2023-07-06T02:04:52.753 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-06T02:04:52.754 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327840 --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327840/orig.config.yaml 2023-07-06T02:04:52.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T02:04:52.896 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327840 branch: wip-rishabh-2023Jul5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: null first_in_suite: false job_id: '7327840' kernel: client: sha1: distro flavor: default kdb: true sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul5 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul5 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b53f236c1077f1d295baafb9e0ca9a4b447019d7 timestamp: 2023-07-05_22:21:20 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-06T02:04:52.897 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa; will attempt to use it 2023-07-06T02:04:52.897 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks 2023-07-06T02:04:52.898 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-06T02:04:52.899 INFO:teuthology.task.internal:Checking packages... 2023-07-06T02:04:52.927 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '79365bb3b803780276d69efb2cc4afe5b160de9e' 2023-07-06T02:04:52.928 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-06T02:04:52.928 INFO:teuthology.packaging:ref: None 2023-07-06T02:04:52.928 INFO:teuthology.packaging:tag: None 2023-07-06T02:04:52.929 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul5 2023-07-06T02:04:52.929 INFO:teuthology.packaging:sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T02:04:52.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul5 2023-07-06T02:04:53.077 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4742.g79365bb3 2023-07-06T02:04:53.078 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-06T02:04:53.085 INFO:teuthology.task.internal:no buildpackages task found 2023-07-06T02:04:53.085 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-06T02:04:53.116 INFO:teuthology.task.internal:Saving configuration 2023-07-06T02:04:53.133 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-06T02:04:53.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T02:04:53.186 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327840', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 01:57:26.244306', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T02:04:53.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327840', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 01:57:26.245889', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T02:04:53.211 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-06T02:04:53.218 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-06T02:04:53.218 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-06T02:04:53.218 INFO:teuthology.run_tasks:Running task console_log... 2023-07-06T02:04:53.326 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb2c647c280>, signals=[15]) 2023-07-06T02:04:53.328 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-06T02:04:53.335 INFO:teuthology.task.internal:Opening connections... 2023-07-06T02:04:53.335 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:04:53.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T02:04:53.406 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:04:53.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T02:04:53.474 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-06T02:04:53.482 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-07-06T02:04:53.503 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-07-06T02:04:53.504 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-07-06T02:04:53.562 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T02:04:53.563 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T02:04:53.563 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel" 2023-07-06T02:04:53.563 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora" 2023-07-06T02:04:53.563 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.6" 2023-07-06T02:04:53.563 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-07-06T02:04:53.563 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T02:04:53.564 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-07-06T02:04:53.564 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T02:04:53.564 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T02:04:53.564 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T02:04:53.564 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T02:04:53.564 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:04:53.564 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T02:04:53.565 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T02:04:53.565 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T02:04:53.565 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T02:04:53.566 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-07-06T02:04:53.592 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2023-07-06T02:04:53.612 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2023-07-06T02:04:53.613 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-07-06T02:04:53.672 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T02:04:53.672 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T02:04:53.672 INFO:teuthology.orchestra.run.smithi172.stdout:ID="rhel" 2023-07-06T02:04:53.672 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="fedora" 2023-07-06T02:04:53.673 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8.6" 2023-07-06T02:04:53.673 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2023-07-06T02:04:53.673 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T02:04:53.673 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-07-06T02:04:53.673 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T02:04:53.674 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T02:04:53.674 INFO:teuthology.orchestra.run.smithi172.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T02:04:53.674 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T02:04:53.674 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:04:53.674 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T02:04:53.674 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T02:04:53.674 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T02:04:53.675 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T02:04:53.675 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-07-06T02:04:53.700 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-06T02:04:53.709 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-06T02:04:53.733 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-06T02:04:53.734 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2023-07-06T02:04:53.736 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2023-07-06T02:04:53.754 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-06T02:04:53.775 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-06T02:04:53.775 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2023-07-06T02:04:53.795 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2023-07-06T02:04:53.823 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-06T02:04:58.889 INFO:teuthology.run_tasks:Running task kernel... 2023-07-06T02:04:58.908 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'} 2023-07-06T02:04:58.909 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-07-06T02:04:58.909 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-07-06T02:04:58.909 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T02:04:58.910 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T02:04:58.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:04:58.929 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2023-07-06T02:04:58.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:04:58.931 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-07-06T02:04:58.989 INFO:teuthology.orchestra.run.smithi172.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-06T02:04:58.990 INFO:teuthology.task.kernel:Running kernel on smithi172: 4.18.0-372.9.1.el8.x86_64 2023-07-06T02:04:58.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum install -y kernel 2023-07-06T02:04:58.991 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-06T02:04:58.992 INFO:teuthology.task.kernel:Running kernel on smithi089: 4.18.0-372.9.1.el8.x86_64 2023-07-06T02:04:58.992 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum install -y kernel 2023-07-06T02:05:00.223 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-07-06T02:05:00.223 INFO:teuthology.orchestra.run.smithi089.stdout:Unable to read consumer identity 2023-07-06T02:05:00.266 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-07-06T02:05:00.266 INFO:teuthology.orchestra.run.smithi172.stdout:Unable to read consumer identity 2023-07-06T02:05:04.147 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T02:05:04.149 INFO:teuthology.orchestra.run.smithi172.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T02:05:07.174 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 16 MB 00:03 2023-07-06T02:05:07.231 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 16 MB 00:03 2023-07-06T02:05:10.631 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 409 kB/s | 24 kB 00:00 2023-07-06T02:05:10.641 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 389 kB/s | 24 kB 00:00 2023-07-06T02:05:11.309 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-06T02:05:11.310 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-06T02:05:11.355 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-07-06T02:05:11.356 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-07-06T02:05:11.356 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-07-06T02:05:11.364 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2023-07-06T02:05:11.365 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2023-07-06T02:05:11.366 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2023-07-06T02:05:11.473 DEBUG:teuthology.orchestra.run.smithi089:> echo no | sudo yum reinstall kernel || true 2023-07-06T02:05:11.482 DEBUG:teuthology.orchestra.run.smithi172:> echo no | sudo yum reinstall kernel || true 2023-07-06T02:05:11.813 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-07-06T02:05:11.813 INFO:teuthology.orchestra.run.smithi172.stdout:Unable to read consumer identity 2023-07-06T02:05:11.842 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-07-06T02:05:11.843 INFO:teuthology.orchestra.run.smithi089.stdout:Unable to read consumer identity 2023-07-06T02:05:12.077 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:02 ago on Thu 06 Jul 2023 02:05:10 AM UTC. 2023-07-06T02:05:12.103 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:02 ago on Thu 06 Jul 2023 02:05:10 AM UTC. 2023-07-06T02:05:12.120 INFO:teuthology.orchestra.run.smithi172.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T02:05:12.143 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T02:05:12.158 INFO:teuthology.orchestra.run.smithi172.stderr:Error: No packages marked for reinstall. 2023-07-06T02:05:12.179 INFO:teuthology.orchestra.run.smithi089.stderr:Error: No packages marked for reinstall. 2023-07-06T02:05:12.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum reinstall -y kernel || true 2023-07-06T02:05:12.225 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum reinstall -y kernel || true 2023-07-06T02:05:12.533 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-07-06T02:05:12.534 INFO:teuthology.orchestra.run.smithi172.stdout:Unable to read consumer identity 2023-07-06T02:05:12.554 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-07-06T02:05:12.554 INFO:teuthology.orchestra.run.smithi089.stdout:Unable to read consumer identity 2023-07-06T02:05:12.793 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:02 ago on Thu 06 Jul 2023 02:05:10 AM UTC. 2023-07-06T02:05:12.808 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:02 ago on Thu 06 Jul 2023 02:05:10 AM UTC. 2023-07-06T02:05:12.832 INFO:teuthology.orchestra.run.smithi172.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T02:05:12.848 INFO:teuthology.orchestra.run.smithi089.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T02:05:12.856 INFO:teuthology.orchestra.run.smithi172.stderr:Error: No packages marked for reinstall. 2023-07-06T02:05:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:Error: No packages marked for reinstall. 2023-07-06T02:05:12.907 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2023-07-06T02:05:12.914 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2023-07-06T02:05:12.987 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-06T02:05:12.987 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-06T02:05:12.988 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-06T02:05:12.988 DEBUG:teuthology.parallel:result is None 2023-07-06T02:05:12.993 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-06T02:05:12.994 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-06T02:05:12.994 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-06T02:05:12.994 DEBUG:teuthology.parallel:result is None 2023-07-06T02:05:12.995 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-06T02:05:13.002 INFO:teuthology.task.internal:Creating test directory... 2023-07-06T02:05:13.003 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T02:05:13.005 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T02:05:13.022 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-06T02:05:13.032 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-06T02:05:13.081 INFO:teuthology.task.internal:Creating archive directory... 2023-07-06T02:05:13.081 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T02:05:13.083 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T02:05:13.110 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-06T02:05:13.120 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-06T02:05:13.121 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T02:05:13.141 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T02:05:13.172 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T02:05:13.175 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T02:05:13.192 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T02:05:13.193 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T02:05:13.195 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-06T02:05:13.208 INFO:teuthology.task.internal:Configuring sudo... 2023-07-06T02:05:13.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T02:05:13.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T02:05:13.262 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-06T02:05:13.271 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-06T02:05:13.272 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T02:05:13.303 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T02:05:13.320 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T02:05:28.412 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T02:05:28.449 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T02:05:33.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T02:05:33.578 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:05:33.579 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T02:05:33.651 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T02:05:38.709 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T02:05:38.975 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T02:05:44.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T02:05:44.053 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:05:44.053 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T02:05:44.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2023-07-06T02:05:44.130 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2023-07-06T02:05:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T02:05:44.242 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T02:05:44.545 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-06T02:05:44.552 INFO:teuthology.task.internal:Starting timer... 2023-07-06T02:05:44.553 INFO:teuthology.run_tasks:Running task pcp... 2023-07-06T02:05:44.606 INFO:teuthology.run_tasks:Running task selinux... 2023-07-06T02:05:44.646 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-07-06T02:05:44.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2023-07-06T02:05:44.711 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2023-07-06T02:05:44.713 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2023-07-06T02:05:44.775 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2023-07-06T02:05:44.777 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-06T02:05:44.777 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2023-07-06T02:05:44.802 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2023-07-06T02:05:44.803 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2023-07-06T02:05:44.824 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2023-07-06T02:05:44.825 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2023-07-06T02:05:44.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T02:05:44.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:05:44.871 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T02:05:44.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:05:44.899 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-06T02:05:44.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2023-07-06T02:05:44.937 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2023-07-06T02:05:44.984 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-06T02:05:44.998 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-06T02:05:44.998 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-06T02:05:45.019 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-06T02:05:45.021 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-06T02:19:08.767 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2023-07-06T02:19:08.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-07-06T02:19:08.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T02:19:08.838 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-07-06T02:19:08.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-07-06T02:19:08.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-07-06T02:19:08.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T02:19:08.979 DEBUG:teuthology.orchestra.run.smithi172:> true 2023-07-06T02:19:09.046 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-07-06T02:19:09.046 INFO:teuthology.run_tasks:Running task clock... 2023-07-06T02:19:09.057 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-06T02:19:09.057 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T02:19:09.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T02:19:09.060 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T02:19:09.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T02:19:09.089 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T02:19:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T02:19:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T02:19:09.133 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T02:19:09.168 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2023-07-06T02:19:09.180 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2023-07-06T02:19:09.195 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T02:19:09.202 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2023-07-06T02:19:09.209 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T02:19:09.214 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2023-07-06T02:19:09.228 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T02:19:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T02:19:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-07-06T02:19:09.298 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T02:19:09.298 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-07-06T02:19:09.298 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.299 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.299 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.299 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.316 INFO:teuthology.orchestra.run.smithi172.stderr:bash: ntpq: command not found 2023-07-06T02:19:09.320 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T02:19:09.320 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2023-07-06T02:19:09.320 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.321 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.321 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.321 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T02:19:09.323 INFO:teuthology.run_tasks:Running task install... 2023-07-06T02:19:09.336 DEBUG:teuthology.task.install:project ceph 2023-07-06T02:19:09.336 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}} 2023-07-06T02:19:09.337 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'} 2023-07-06T02:19:09.337 INFO:teuthology.task.install:Using flavor: default 2023-07-06T02:19:09.348 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-06T02:19:09.349 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-06T02:19:09.349 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T02:19:09.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T02:19:09.352 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T02:19:09.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T02:19:09.569 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T02:19:09.570 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T02:19:09.583 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T02:19:09.583 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T02:19:09.944 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T02:19:09.945 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:19:09.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T02:19:09.956 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T02:19:09.957 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:19:09.957 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T02:19:09.974 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T02:19:09.975 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T02:19:09.986 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T02:19:09.986 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T02:19:10.046 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T02:19:10.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T02:19:10.125 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T02:19:10.134 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T02:19:10.192 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2023-07-06T02:19:10.194 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2023-07-06T02:19:10.200 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2023-07-06T02:19:10.202 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2023-07-06T02:19:10.565 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-07-06T02:19:10.575 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-07-06T02:19:10.834 INFO:teuthology.orchestra.run.smithi089.stdout:44 files removed 2023-07-06T02:19:10.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T02:19:10.940 INFO:teuthology.orchestra.run.smithi172.stdout:44 files removed 2023-07-06T02:19:10.988 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T02:19:11.211 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-07-06T02:19:11.321 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-07-06T02:19:12.351 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 103 kB/s | 81 kB 00:00 2023-07-06T02:19:12.508 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 96 kB/s | 81 kB 00:00 2023-07-06T02:19:13.160 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-07-06T02:19:13.325 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-07-06T02:19:13.886 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-07-06T02:19:14.092 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 2.0 kB/s | 1.5 kB 00:00 2023-07-06T02:19:16.344 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-07-06T02:19:16.907 INFO:teuthology.orchestra.run.smithi172.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-06T02:19:17.354 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-07-06T02:19:20.812 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 262 kB/s | 24 kB 00:00 2023-07-06T02:19:21.814 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 53 MB 00:00 2023-07-06T02:19:26.118 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 16 MB 00:09 2023-07-06T02:19:29.568 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 433 kB/s | 24 kB 00:00 2023-07-06T02:19:29.890 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-07-06T02:19:30.536 INFO:teuthology.orchestra.run.smithi172.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-07-06T02:19:38.595 INFO:teuthology.orchestra.run.smithi172.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-07-06T02:19:40.848 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T02:19:40.849 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T02:19:40.862 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T02:19:40.864 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T02:19:40.865 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T02:19:40.866 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T02:19:40.868 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T02:19:40.871 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T02:19:40.873 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T02:19:41.008 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-07-06T02:19:41.020 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-07-06T02:19:41.021 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-07-06T02:19:41.021 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-07-06T02:19:41.021 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-07-06T02:19:41.021 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T02:19:41.021 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T02:19:41.021 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T02:19:41.022 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T02:19:41.022 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T02:19:41.022 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T02:19:41.022 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T02:19:41.022 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T02:19:41.022 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T02:19:41.022 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T02:19:41.023 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T02:19:41.023 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T02:19:41.023 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T02:19:41.023 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T02:19:41.023 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T02:19:41.023 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T02:19:41.023 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T02:19:41.024 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T02:19:41.024 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T02:19:41.024 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T02:19:41.024 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T02:19:41.024 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T02:19:41.024 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T02:19:41.024 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T02:19:41.025 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T02:19:41.025 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T02:19:41.025 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T02:19:41.025 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T02:19:41.025 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T02:19:41.025 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T02:19:41.025 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-07-06T02:19:41.026 INFO:teuthology.orchestra.run.smithi089.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T02:19:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T02:19:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T02:19:41.027 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-07-06T02:19:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T02:19:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T02:19:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T02:19:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T02:19:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T02:19:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T02:19:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T02:19:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T02:19:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T02:19:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T02:19:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T02:19:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T02:19:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T02:19:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T02:19:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T02:19:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T02:19:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T02:19:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T02:19:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T02:19:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T02:19:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T02:19:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T02:19:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T02:19:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T02:19:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T02:19:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T02:19:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T02:19:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T02:19:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T02:19:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T02:19:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T02:19:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T02:19:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T02:19:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T02:19:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T02:19:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T02:19:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T02:19:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T02:19:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T02:19:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T02:19:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T02:19:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T02:19:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T02:19:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T02:19:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T02:19:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T02:19:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T02:19:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T02:19:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T02:19:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T02:19:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T02:19:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T02:19:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T02:19:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T02:19:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T02:19:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T02:19:41.040 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling module streams: 2023-07-06T02:19:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql 10 2023-07-06T02:19:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:19:41.040 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-07-06T02:19:41.040 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-07-06T02:19:41.040 INFO:teuthology.orchestra.run.smithi089.stdout:Install 128 Packages 2023-07-06T02:19:41.040 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 3 Packages 2023-07-06T02:19:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:19:41.041 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 205 M 2023-07-06T02:19:41.041 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-07-06T02:19:41.474 INFO:teuthology.orchestra.run.smithi089.stdout:(1/131): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-07-06T02:19:42.199 INFO:teuthology.orchestra.run.smithi089.stdout:(2/131): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 1.1 MB/s | 847 kB 00:00 2023-07-06T02:19:42.307 INFO:teuthology.orchestra.run.smithi089.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.5 MB/s | 165 kB 00:00 2023-07-06T02:19:43.166 INFO:teuthology.orchestra.run.smithi089.stdout:(4/131): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 2.7 MB/s | 2.3 MB 00:00 2023-07-06T02:19:43.708 INFO:teuthology.orchestra.run.smithi089.stdout:(5/131): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 2.9 MB/s | 1.5 MB 00:00 2023-07-06T02:19:45.425 INFO:teuthology.orchestra.run.smithi089.stdout:(6/131): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 2.7 MB/s | 4.6 MB 00:01 2023-07-06T02:19:49.463 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T02:19:49.464 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T02:19:49.478 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T02:19:49.480 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T02:19:49.481 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T02:19:49.482 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T02:19:49.484 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T02:19:49.487 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T02:19:49.489 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T02:19:49.619 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2023-07-06T02:19:49.631 INFO:teuthology.orchestra.run.smithi172.stdout:=========================================================================================================================== 2023-07-06T02:19:49.632 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2023-07-06T02:19:49.632 INFO:teuthology.orchestra.run.smithi172.stdout:=========================================================================================================================== 2023-07-06T02:19:49.632 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2023-07-06T02:19:49.632 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T02:19:49.632 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T02:19:49.632 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T02:19:49.632 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T02:19:49.633 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T02:19:49.633 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T02:19:49.633 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T02:19:49.633 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T02:19:49.633 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T02:19:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T02:19:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T02:19:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T02:19:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T02:19:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T02:19:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T02:19:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T02:19:49.635 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T02:19:49.635 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T02:19:49.635 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T02:19:49.636 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T02:19:49.636 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T02:19:49.636 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T02:19:49.637 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T02:19:49.637 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T02:19:49.638 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T02:19:49.638 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T02:19:49.638 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T02:19:49.638 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T02:19:49.639 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T02:19:49.639 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T02:19:49.639 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T02:19:49.640 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T02:19:49.640 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T02:19:49.640 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T02:19:49.640 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T02:19:49.641 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T02:19:49.641 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T02:19:49.641 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2023-07-06T02:19:49.642 INFO:teuthology.orchestra.run.smithi172.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T02:19:49.642 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T02:19:49.642 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T02:19:49.642 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2023-07-06T02:19:49.643 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T02:19:49.643 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T02:19:49.643 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T02:19:49.643 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T02:19:49.644 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T02:19:49.644 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T02:19:49.644 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T02:19:49.644 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T02:19:49.644 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T02:19:49.644 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T02:19:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T02:19:49.646 INFO:teuthology.orchestra.run.smithi172.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T02:19:49.646 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T02:19:49.646 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T02:19:49.646 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T02:19:49.646 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T02:19:49.646 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T02:19:49.646 INFO:teuthology.orchestra.run.smithi172.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T02:19:49.647 INFO:teuthology.orchestra.run.smithi172.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T02:19:49.647 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T02:19:49.648 INFO:teuthology.orchestra.run.smithi172.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T02:19:49.648 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T02:19:49.648 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T02:19:49.648 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T02:19:49.649 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T02:19:49.649 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T02:19:49.649 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T02:19:49.650 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T02:19:49.650 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T02:19:49.650 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T02:19:49.650 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T02:19:49.651 INFO:teuthology.orchestra.run.smithi172.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T02:19:49.651 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T02:19:49.651 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T02:19:49.651 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T02:19:49.652 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T02:19:49.652 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T02:19:49.652 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T02:19:49.652 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T02:19:49.652 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T02:19:49.653 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T02:19:49.653 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T02:19:49.653 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T02:19:49.654 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T02:19:49.654 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T02:19:49.654 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T02:19:49.654 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T02:19:49.654 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T02:19:49.654 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T02:19:49.655 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T02:19:49.655 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T02:19:49.655 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T02:19:49.655 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T02:19:49.655 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T02:19:49.655 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T02:19:49.655 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T02:19:49.656 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T02:19:49.657 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T02:19:49.657 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T02:19:49.657 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T02:19:49.657 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T02:19:49.657 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T02:19:49.657 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T02:19:49.657 INFO:teuthology.orchestra.run.smithi172.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T02:19:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T02:19:49.659 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T02:19:49.659 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T02:19:49.659 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T02:19:49.659 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T02:19:49.659 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T02:19:49.659 INFO:teuthology.orchestra.run.smithi172.stdout:Enabling module streams: 2023-07-06T02:19:49.659 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql 10 2023-07-06T02:19:49.660 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:19:49.660 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2023-07-06T02:19:49.660 INFO:teuthology.orchestra.run.smithi172.stdout:=========================================================================================================================== 2023-07-06T02:19:49.660 INFO:teuthology.orchestra.run.smithi172.stdout:Install 128 Packages 2023-07-06T02:19:49.660 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 3 Packages 2023-07-06T02:19:49.660 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:19:49.661 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 205 M 2023-07-06T02:19:49.661 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2023-07-06T02:19:50.037 INFO:teuthology.orchestra.run.smithi172.stdout:(1/131): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-07-06T02:19:50.316 INFO:teuthology.orchestra.run.smithi089.stdout:(7/131): ceph-base-18.0.0-4742.g79365bb3.el8.x8 596 kB/s | 5.4 MB 00:09 2023-07-06T02:19:51.037 INFO:teuthology.orchestra.run.smithi172.stdout:(2/131): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 847 kB/s | 847 kB 00:00 2023-07-06T02:19:51.196 INFO:teuthology.orchestra.run.smithi172.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.0 MB/s | 165 kB 00:00 2023-07-06T02:19:52.588 INFO:teuthology.orchestra.run.smithi172.stdout:(4/131): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 1.6 MB/s | 2.3 MB 00:01 2023-07-06T02:19:53.205 INFO:teuthology.orchestra.run.smithi172.stdout:(5/131): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 2.5 MB/s | 1.5 MB 00:00 2023-07-06T02:19:54.633 INFO:teuthology.orchestra.run.smithi089.stdout:(8/131): ceph-osd-18.0.0-4742.g79365bb3.el8.x86 1.9 MB/s | 18 MB 00:09 2023-07-06T02:19:54.750 INFO:teuthology.orchestra.run.smithi089.stdout:(9/131): ceph-selinux-18.0.0-4742.g79365bb3.el8 209 kB/s | 24 kB 00:00 2023-07-06T02:19:55.113 INFO:teuthology.orchestra.run.smithi172.stdout:(6/131): ceph-base-18.0.0-4742.g79365bb3.el8.x8 1.0 MB/s | 5.4 MB 00:05 2023-07-06T02:19:55.763 INFO:teuthology.orchestra.run.smithi172.stdout:(7/131): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 1.8 MB/s | 4.6 MB 00:02 2023-07-06T02:20:00.788 INFO:teuthology.orchestra.run.smithi172.stdout:(8/131): ceph-radosgw-18.0.0-4742.g79365bb3.el8 2.7 MB/s | 13 MB 00:05 2023-07-06T02:20:00.880 INFO:teuthology.orchestra.run.smithi172.stdout:(9/131): ceph-selinux-18.0.0-4742.g79365bb3.el8 266 kB/s | 24 kB 00:00 2023-07-06T02:20:02.024 INFO:teuthology.orchestra.run.smithi089.stdout:(10/131): ceph-radosgw-18.0.0-4742.g79365bb3.el 1.2 MB/s | 13 MB 00:11 2023-07-06T02:20:02.374 INFO:teuthology.orchestra.run.smithi089.stdout:(11/131): cephfs-mirror-18.0.0-4742.g79365bb3.e 738 kB/s | 258 kB 00:00 2023-07-06T02:20:02.499 INFO:teuthology.orchestra.run.smithi089.stdout:(12/131): libcephfs-devel-18.0.0-4742.g79365bb3 248 kB/s | 31 kB 00:00 2023-07-06T02:20:03.191 INFO:teuthology.orchestra.run.smithi089.stdout:(13/131): libcephfs2-18.0.0-4742.g79365bb3.el8. 1.1 MB/s | 770 kB 00:00 2023-07-06T02:20:03.483 INFO:teuthology.orchestra.run.smithi089.stdout:(14/131): libcephsqlite-18.0.0-4742.g79365bb3.e 470 kB/s | 137 kB 00:00 2023-07-06T02:20:03.725 INFO:teuthology.orchestra.run.smithi089.stdout:(15/131): librados-devel-18.0.0-4742.g79365bb3. 574 kB/s | 139 kB 00:00 2023-07-06T02:20:04.276 INFO:teuthology.orchestra.run.smithi089.stdout:(16/131): libradosstriper1-18.0.0-4742.g79365bb 864 kB/s | 475 kB 00:00 2023-07-06T02:20:05.609 INFO:teuthology.orchestra.run.smithi089.stdout:(17/131): ceph-common-18.0.0-4742.g79365bb3.el8 920 kB/s | 22 MB 00:24 2023-07-06T02:20:05.776 INFO:teuthology.orchestra.run.smithi089.stdout:(18/131): python3-ceph-argparse-18.0.0-4742.g79 269 kB/s | 45 kB 00:00 2023-07-06T02:20:05.943 INFO:teuthology.orchestra.run.smithi089.stdout:(19/131): python3-ceph-common-18.0.0-4742.g7936 713 kB/s | 119 kB 00:00 2023-07-06T02:20:06.177 INFO:teuthology.orchestra.run.smithi089.stdout:(20/131): python3-cephfs-18.0.0-4742.g79365bb3. 844 kB/s | 197 kB 00:00 2023-07-06T02:20:06.560 INFO:teuthology.orchestra.run.smithi089.stdout:(21/131): python3-rados-18.0.0-4742.g79365bb3.e 966 kB/s | 370 kB 00:00 2023-07-06T02:20:06.927 INFO:teuthology.orchestra.run.smithi089.stdout:(22/131): python3-rbd-18.0.0-4742.g79365bb3.el8 995 kB/s | 364 kB 00:00 2023-07-06T02:20:07.094 INFO:teuthology.orchestra.run.smithi089.stdout:(23/131): python3-rgw-18.0.0-4742.g79365bb3.el8 640 kB/s | 107 kB 00:00 2023-07-06T02:20:07.269 INFO:teuthology.orchestra.run.smithi089.stdout:(24/131): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 539 kB/s | 94 kB 00:00 2023-07-06T02:20:09.988 INFO:teuthology.orchestra.run.smithi172.stdout:(10/131): ceph-osd-18.0.0-4742.g79365bb3.el8.x8 1.2 MB/s | 18 MB 00:14 2023-07-06T02:20:10.112 INFO:teuthology.orchestra.run.smithi089.stdout:(25/131): rbd-mirror-18.0.0-4742.g79365bb3.el8. 1.2 MB/s | 3.5 MB 00:02 2023-07-06T02:20:10.187 INFO:teuthology.orchestra.run.smithi172.stdout:(11/131): cephfs-mirror-18.0.0-4742.g79365bb3.e 1.3 MB/s | 258 kB 00:00 2023-07-06T02:20:10.278 INFO:teuthology.orchestra.run.smithi089.stdout:(26/131): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 1.1 MB/s | 179 kB 00:00 2023-07-06T02:20:10.304 INFO:teuthology.orchestra.run.smithi172.stdout:(12/131): libcephfs-devel-18.0.0-4742.g79365bb3 266 kB/s | 31 kB 00:00 2023-07-06T02:20:10.379 INFO:teuthology.orchestra.run.smithi089.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4742.g 230 kB/s | 23 kB 00:00 2023-07-06T02:20:10.545 INFO:teuthology.orchestra.run.smithi089.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4742.g79365bb 774 kB/s | 129 kB 00:00 2023-07-06T02:20:10.667 INFO:teuthology.orchestra.run.smithi089.stdout:(29/131): librgw2-18.0.0-4742.g79365bb3.el8.x86 1.1 MB/s | 6.7 MB 00:06 2023-07-06T02:20:10.871 INFO:teuthology.orchestra.run.smithi172.stdout:(13/131): libcephfs2-18.0.0-4742.g79365bb3.el8. 1.3 MB/s | 770 kB 00:00 2023-07-06T02:20:11.005 INFO:teuthology.orchestra.run.smithi172.stdout:(14/131): libcephsqlite-18.0.0-4742.g79365bb3.e 1.0 MB/s | 137 kB 00:00 2023-07-06T02:20:11.130 INFO:teuthology.orchestra.run.smithi172.stdout:(15/131): librados-devel-18.0.0-4742.g79365bb3. 1.1 MB/s | 139 kB 00:00 2023-07-06T02:20:11.447 INFO:teuthology.orchestra.run.smithi172.stdout:(16/131): libradosstriper1-18.0.0-4742.g79365bb 1.5 MB/s | 475 kB 00:00 2023-07-06T02:20:12.304 INFO:teuthology.orchestra.run.smithi089.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4742.g79365 996 kB/s | 1.7 MB 00:01 2023-07-06T02:20:12.538 INFO:teuthology.orchestra.run.smithi089.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4742.g79 1.0 MB/s | 240 kB 00:00 2023-07-06T02:20:12.638 INFO:teuthology.orchestra.run.smithi089.stdout:(32/131): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 475 kB/s | 47 kB 00:00 2023-07-06T02:20:12.730 INFO:teuthology.orchestra.run.smithi089.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4742.g7 158 kB/s | 15 kB 00:00 2023-07-06T02:20:13.047 INFO:teuthology.orchestra.run.smithi089.stdout:(34/131): ceph-volume-18.0.0-4742.g79365bb3.el8 815 kB/s | 258 kB 00:00 2023-07-06T02:20:13.214 INFO:teuthology.orchestra.run.smithi089.stdout:(35/131): cephadm-18.0.0-4742.g79365bb3.el8.noa 1.0 MB/s | 166 kB 00:00 2023-07-06T02:20:13.247 INFO:teuthology.orchestra.run.smithi172.stdout:(17/131): ceph-common-18.0.0-4742.g79365bb3.el8 960 kB/s | 22 MB 00:23 2023-07-06T02:20:13.314 INFO:teuthology.orchestra.run.smithi089.stdout:(36/131): cephfs-top-18.0.0-4742.g79365bb3.el8. 179 kB/s | 18 kB 00:00 2023-07-06T02:20:13.339 INFO:teuthology.orchestra.run.smithi172.stdout:(18/131): python3-ceph-argparse-18.0.0-4742.g79 489 kB/s | 45 kB 00:00 2023-07-06T02:20:13.514 INFO:teuthology.orchestra.run.smithi172.stdout:(19/131): python3-ceph-common-18.0.0-4742.g7936 679 kB/s | 119 kB 00:00 2023-07-06T02:20:13.631 INFO:teuthology.orchestra.run.smithi089.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 969 kB/s | 306 kB 00:00 2023-07-06T02:20:13.681 INFO:teuthology.orchestra.run.smithi172.stdout:(20/131): python3-cephfs-18.0.0-4742.g79365bb3. 1.2 MB/s | 197 kB 00:00 2023-07-06T02:20:13.706 INFO:teuthology.orchestra.run.smithi089.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 785 kB/s | 59 kB 00:00 2023-07-06T02:20:13.782 INFO:teuthology.orchestra.run.smithi089.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-07-06T02:20:13.915 INFO:teuthology.orchestra.run.smithi089.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-07-06T02:20:13.981 INFO:teuthology.orchestra.run.smithi172.stdout:(21/131): python3-rados-18.0.0-4742.g79365bb3.e 1.2 MB/s | 370 kB 00:00 2023-07-06T02:20:13.990 INFO:teuthology.orchestra.run.smithi089.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 582 kB/s | 44 kB 00:00 2023-07-06T02:20:14.066 INFO:teuthology.orchestra.run.smithi089.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-07-06T02:20:14.141 INFO:teuthology.orchestra.run.smithi089.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 440 kB/s | 33 kB 00:00 2023-07-06T02:20:14.216 INFO:teuthology.orchestra.run.smithi089.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 218 kB/s | 16 kB 00:00 2023-07-06T02:20:14.273 INFO:teuthology.orchestra.run.smithi172.stdout:(22/131): python3-rbd-18.0.0-4742.g79365bb3.el8 1.2 MB/s | 364 kB 00:00 2023-07-06T02:20:14.292 INFO:teuthology.orchestra.run.smithi089.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-07-06T02:20:14.423 INFO:teuthology.orchestra.run.smithi172.stdout:(23/131): python3-rgw-18.0.0-4742.g79365bb3.el8 713 kB/s | 107 kB 00:00 2023-07-06T02:20:14.467 INFO:teuthology.orchestra.run.smithi089.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-07-06T02:20:14.515 INFO:teuthology.orchestra.run.smithi172.stdout:(24/131): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 1.0 MB/s | 94 kB 00:00 2023-07-06T02:20:14.909 INFO:teuthology.orchestra.run.smithi089.stdout:(47/131): ceph-test-18.0.0-4742.g79365bb3.el8.x 2.4 MB/s | 48 MB 00:20 2023-07-06T02:20:14.934 INFO:teuthology.orchestra.run.smithi089.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 174 kB/s | 81 kB 00:00 2023-07-06T02:20:15.010 INFO:teuthology.orchestra.run.smithi089.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-07-06T02:20:15.043 INFO:teuthology.orchestra.run.smithi089.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-06T02:20:15.099 INFO:teuthology.orchestra.run.smithi172.stdout:(25/131): librgw2-18.0.0-4742.g79365bb3.el8.x86 1.8 MB/s | 6.7 MB 00:03 2023-07-06T02:20:15.225 INFO:teuthology.orchestra.run.smithi089.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 268 kB/s | 49 kB 00:00 2023-07-06T02:20:15.283 INFO:teuthology.orchestra.run.smithi172.stdout:(26/131): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 980 kB/s | 179 kB 00:00 2023-07-06T02:20:15.319 INFO:teuthology.orchestra.run.smithi089.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 3.8 MB/s | 1.2 MB 00:00 2023-07-06T02:20:15.369 INFO:teuthology.orchestra.run.smithi089.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 407 kB/s | 59 kB 00:00 2023-07-06T02:20:15.403 INFO:teuthology.orchestra.run.smithi089.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 722 kB/s | 60 kB 00:00 2023-07-06T02:20:15.425 INFO:teuthology.orchestra.run.smithi172.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4742.g 162 kB/s | 23 kB 00:00 2023-07-06T02:20:15.495 INFO:teuthology.orchestra.run.smithi089.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-07-06T02:20:15.528 INFO:teuthology.orchestra.run.smithi089.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-07-06T02:20:15.567 INFO:teuthology.orchestra.run.smithi172.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4742.g79365bb 910 kB/s | 129 kB 00:00 2023-07-06T02:20:15.579 INFO:teuthology.orchestra.run.smithi089.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 403 kB/s | 34 kB 00:00 2023-07-06T02:20:15.629 INFO:teuthology.orchestra.run.smithi089.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-07-06T02:20:15.671 INFO:teuthology.orchestra.run.smithi089.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 670 kB/s | 61 kB 00:00 2023-07-06T02:20:15.721 INFO:teuthology.orchestra.run.smithi089.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-07-06T02:20:15.763 INFO:teuthology.orchestra.run.smithi089.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2023-07-06T02:20:15.813 INFO:teuthology.orchestra.run.smithi089.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-07-06T02:20:15.847 INFO:teuthology.orchestra.run.smithi089.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 330 kB/s | 27 kB 00:00 2023-07-06T02:20:15.914 INFO:teuthology.orchestra.run.smithi089.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-07-06T02:20:15.948 INFO:teuthology.orchestra.run.smithi089.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-07-06T02:20:16.006 INFO:teuthology.orchestra.run.smithi089.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 667 kB/s | 61 kB 00:00 2023-07-06T02:20:16.032 INFO:teuthology.orchestra.run.smithi089.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-07-06T02:20:16.107 INFO:teuthology.orchestra.run.smithi089.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-07-06T02:20:16.132 INFO:teuthology.orchestra.run.smithi089.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-06T02:20:16.191 INFO:teuthology.orchestra.run.smithi089.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 804 kB/s | 47 kB 00:00 2023-07-06T02:20:16.233 INFO:teuthology.orchestra.run.smithi089.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-06T02:20:16.275 INFO:teuthology.orchestra.run.smithi089.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-06T02:20:16.317 INFO:teuthology.orchestra.run.smithi089.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-06T02:20:16.359 INFO:teuthology.orchestra.run.smithi089.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2023-07-06T02:20:16.384 INFO:teuthology.orchestra.run.smithi089.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 719 kB/s | 48 kB 00:00 2023-07-06T02:20:16.434 INFO:teuthology.orchestra.run.smithi089.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 989 kB/s | 43 kB 00:00 2023-07-06T02:20:16.460 INFO:teuthology.orchestra.run.smithi089.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 867 kB/s | 87 kB 00:00 2023-07-06T02:20:16.485 INFO:teuthology.orchestra.run.smithi089.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-06T02:20:16.519 INFO:teuthology.orchestra.run.smithi089.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-06T02:20:16.534 INFO:teuthology.orchestra.run.smithi172.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4742.g79365 1.8 MB/s | 1.7 MB 00:00 2023-07-06T02:20:16.552 INFO:teuthology.orchestra.run.smithi089.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.1 MB/s | 560 kB 00:00 2023-07-06T02:20:16.577 INFO:teuthology.orchestra.run.smithi089.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 318 kB/s | 19 kB 00:00 2023-07-06T02:20:16.603 INFO:teuthology.orchestra.run.smithi089.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-06T02:20:16.636 INFO:teuthology.orchestra.run.smithi089.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-07-06T02:20:16.662 INFO:teuthology.orchestra.run.smithi089.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-07-06T02:20:16.695 INFO:teuthology.orchestra.run.smithi089.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-06T02:20:16.746 INFO:teuthology.orchestra.run.smithi089.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-07-06T02:20:16.787 INFO:teuthology.orchestra.run.smithi089.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-06T02:20:16.838 INFO:teuthology.orchestra.run.smithi089.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2023-07-06T02:20:16.855 INFO:teuthology.orchestra.run.smithi089.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-07-06T02:20:16.880 INFO:teuthology.orchestra.run.smithi089.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-07-06T02:20:16.901 INFO:teuthology.orchestra.run.smithi172.stdout:(30/131): rbd-mirror-18.0.0-4742.g79365bb3.el8. 1.4 MB/s | 3.5 MB 00:02 2023-07-06T02:20:16.905 INFO:teuthology.orchestra.run.smithi089.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-06T02:20:16.931 INFO:teuthology.orchestra.run.smithi089.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-06T02:20:16.956 INFO:teuthology.orchestra.run.smithi089.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-07-06T02:20:16.989 INFO:teuthology.orchestra.run.smithi089.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-06T02:20:17.015 INFO:teuthology.orchestra.run.smithi089.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-06T02:20:17.040 INFO:teuthology.orchestra.run.smithi089.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-06T02:20:17.082 INFO:teuthology.orchestra.run.smithi089.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-06T02:20:17.116 INFO:teuthology.orchestra.run.smithi089.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-06T02:20:17.134 INFO:teuthology.orchestra.run.smithi172.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4742.g79 1.0 MB/s | 240 kB 00:00 2023-07-06T02:20:17.158 INFO:teuthology.orchestra.run.smithi089.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-06T02:20:17.200 INFO:teuthology.orchestra.run.smithi089.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-06T02:20:17.233 INFO:teuthology.orchestra.run.smithi089.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-07-06T02:20:17.235 INFO:teuthology.orchestra.run.smithi172.stdout:(32/131): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 474 kB/s | 47 kB 00:00 2023-07-06T02:20:17.284 INFO:teuthology.orchestra.run.smithi089.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-07-06T02:20:17.325 INFO:teuthology.orchestra.run.smithi089.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-07-06T02:20:17.335 INFO:teuthology.orchestra.run.smithi172.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4742.g7 145 kB/s | 15 kB 00:00 2023-07-06T02:20:17.367 INFO:teuthology.orchestra.run.smithi089.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-06T02:20:17.410 INFO:teuthology.orchestra.run.smithi089.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-06T02:20:17.560 INFO:teuthology.orchestra.run.smithi089.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 25 MB/s | 14 MB 00:00 2023-07-06T02:20:17.593 INFO:teuthology.orchestra.run.smithi172.stdout:(34/131): ceph-volume-18.0.0-4742.g79365bb3.el8 1.0 MB/s | 258 kB 00:00 2023-07-06T02:20:17.594 INFO:teuthology.orchestra.run.smithi089.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 3.8 MB/s | 713 kB 00:00 2023-07-06T02:20:17.627 INFO:teuthology.orchestra.run.smithi089.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 793 kB/s | 27 kB 00:00 2023-07-06T02:20:17.669 INFO:teuthology.orchestra.run.smithi089.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-06T02:20:17.703 INFO:teuthology.orchestra.run.smithi089.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-07-06T02:20:17.777 INFO:teuthology.orchestra.run.smithi172.stdout:(35/131): cephadm-18.0.0-4742.g79365bb3.el8.noa 908 kB/s | 166 kB 00:00 2023-07-06T02:20:17.778 INFO:teuthology.orchestra.run.smithi089.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-07-06T02:20:17.820 INFO:teuthology.orchestra.run.smithi089.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-07-06T02:20:17.862 INFO:teuthology.orchestra.run.smithi089.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-06T02:20:17.877 INFO:teuthology.orchestra.run.smithi172.stdout:(36/131): cephfs-top-18.0.0-4742.g79365bb3.el8. 180 kB/s | 18 kB 00:00 2023-07-06T02:20:17.920 INFO:teuthology.orchestra.run.smithi089.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-07-06T02:20:17.946 INFO:teuthology.orchestra.run.smithi089.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 764 kB/s | 64 kB 00:00 2023-07-06T02:20:17.971 INFO:teuthology.orchestra.run.smithi089.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-06T02:20:17.996 INFO:teuthology.orchestra.run.smithi089.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-07-06T02:20:18.022 INFO:teuthology.orchestra.run.smithi089.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-06T02:20:18.055 INFO:teuthology.orchestra.run.smithi089.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-06T02:20:18.097 INFO:teuthology.orchestra.run.smithi089.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-06T02:20:18.221 INFO:teuthology.orchestra.run.smithi089.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 24 MB/s | 4.9 MB 00:00 2023-07-06T02:20:18.264 INFO:teuthology.orchestra.run.smithi089.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.6 MB/s | 202 kB 00:00 2023-07-06T02:20:18.327 INFO:teuthology.orchestra.run.smithi172.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 681 kB/s | 306 kB 00:00 2023-07-06T02:20:18.340 INFO:teuthology.orchestra.run.smithi089.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-07-06T02:20:18.373 INFO:teuthology.orchestra.run.smithi089.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-07-06T02:20:18.424 INFO:teuthology.orchestra.run.smithi089.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-07-06T02:20:18.453 INFO:teuthology.orchestra.run.smithi172.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 470 kB/s | 59 kB 00:00 2023-07-06T02:20:18.516 INFO:teuthology.orchestra.run.smithi089.stdout:(126/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-07-06T02:20:18.549 INFO:teuthology.orchestra.run.smithi089.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-06T02:20:18.570 INFO:teuthology.orchestra.run.smithi172.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 642 kB/s | 75 kB 00:00 2023-07-06T02:20:18.800 INFO:teuthology.orchestra.run.smithi089.stdout:(128/131): ceph-mgr-diskprediction-local-18.0.0 931 kB/s | 7.4 MB 00:08 2023-07-06T02:20:18.803 INFO:teuthology.orchestra.run.smithi172.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-07-06T02:20:18.850 INFO:teuthology.orchestra.run.smithi089.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-06T02:20:18.912 INFO:teuthology.orchestra.run.smithi172.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-07-06T02:20:19.054 INFO:teuthology.orchestra.run.smithi172.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-07-06T02:20:19.188 INFO:teuthology.orchestra.run.smithi172.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 247 kB/s | 33 kB 00:00 2023-07-06T02:20:19.338 INFO:teuthology.orchestra.run.smithi172.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 109 kB/s | 16 kB 00:00 2023-07-06T02:20:19.455 INFO:teuthology.orchestra.run.smithi172.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-07-06T02:20:19.622 INFO:teuthology.orchestra.run.smithi172.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.3 MB/s | 384 kB 00:00 2023-07-06T02:20:19.731 INFO:teuthology.orchestra.run.smithi172.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 751 kB/s | 81 kB 00:00 2023-07-06T02:20:19.848 INFO:teuthology.orchestra.run.smithi172.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-07-06T02:20:19.956 INFO:teuthology.orchestra.run.smithi172.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-07-06T02:20:20.190 INFO:teuthology.orchestra.run.smithi172.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-07-06T02:20:20.332 INFO:teuthology.orchestra.run.smithi172.stdout:(51/131): ceph-mgr-diskprediction-local-18.0.0- 1.9 MB/s | 7.4 MB 00:03 2023-07-06T02:20:20.357 INFO:teuthology.orchestra.run.smithi172.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 291 kB/s | 49 kB 00:00 2023-07-06T02:20:20.474 INFO:teuthology.orchestra.run.smithi172.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 515 kB/s | 60 kB 00:00 2023-07-06T02:20:20.608 INFO:teuthology.orchestra.run.smithi172.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-07-06T02:20:20.683 INFO:teuthology.orchestra.run.smithi172.stdout:(55/131): python3-more-itertools-7.2.0-3.el8.no 167 kB/s | 59 kB 00:00 2023-07-06T02:20:20.708 INFO:teuthology.orchestra.run.smithi172.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-07-06T02:20:20.775 INFO:teuthology.orchestra.run.smithi172.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2023-07-06T02:20:20.859 INFO:teuthology.orchestra.run.smithi172.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-07-06T02:20:20.951 INFO:teuthology.orchestra.run.smithi172.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 351 kB/s | 61 kB 00:00 2023-07-06T02:20:20.976 INFO:teuthology.orchestra.run.smithi172.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2023-07-06T02:20:21.043 INFO:teuthology.orchestra.run.smithi172.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 261 kB/s | 24 kB 00:00 2023-07-06T02:20:21.077 INFO:teuthology.orchestra.run.smithi172.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 292 kB/s | 29 kB 00:00 2023-07-06T02:20:21.135 INFO:teuthology.orchestra.run.smithi172.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-07-06T02:20:21.186 INFO:teuthology.orchestra.run.smithi172.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-07-06T02:20:21.319 INFO:teuthology.orchestra.run.smithi172.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 459 kB/s | 61 kB 00:00 2023-07-06T02:20:21.411 INFO:teuthology.orchestra.run.smithi172.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 932 kB/s | 85 kB 00:00 2023-07-06T02:20:21.492 INFO:teuthology.orchestra.run.smithi089.stdout:(130/131): librbd1-18.0.0-4742.g79365bb3.el8.x8 1.2 MB/s | 3.6 MB 00:02 2023-07-06T02:20:21.495 INFO:teuthology.orchestra.run.smithi172.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 276 kB/s | 23 kB 00:00 2023-07-06T02:20:21.520 INFO:teuthology.orchestra.run.smithi172.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 653 kB/s | 251 kB 00:00 2023-07-06T02:20:21.545 INFO:teuthology.orchestra.run.smithi172.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-06T02:20:21.604 INFO:teuthology.orchestra.run.smithi172.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 807 kB/s | 47 kB 00:00 2023-07-06T02:20:21.646 INFO:teuthology.orchestra.run.smithi172.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-06T02:20:21.688 INFO:teuthology.orchestra.run.smithi172.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-06T02:20:21.730 INFO:teuthology.orchestra.run.smithi172.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-06T02:20:21.772 INFO:teuthology.orchestra.run.smithi172.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 6.3 MB/s | 1.7 MB 00:00 2023-07-06T02:20:21.797 INFO:teuthology.orchestra.run.smithi172.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 719 kB/s | 48 kB 00:00 2023-07-06T02:20:21.839 INFO:teuthology.orchestra.run.smithi172.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.2 MB/s | 43 kB 00:00 2023-07-06T02:20:21.865 INFO:teuthology.orchestra.run.smithi172.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 944 kB/s | 87 kB 00:00 2023-07-06T02:20:21.890 INFO:teuthology.orchestra.run.smithi172.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-06T02:20:21.915 INFO:teuthology.orchestra.run.smithi172.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-07-06T02:20:21.949 INFO:teuthology.orchestra.run.smithi172.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-07-06T02:20:21.967 INFO:teuthology.orchestra.run.smithi172.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 371 kB/s | 19 kB 00:00 2023-07-06T02:20:21.991 INFO:teuthology.orchestra.run.smithi172.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-07-06T02:20:22.025 INFO:teuthology.orchestra.run.smithi172.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-07-06T02:20:22.042 INFO:teuthology.orchestra.run.smithi172.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 357 kB/s | 18 kB 00:00 2023-07-06T02:20:22.075 INFO:teuthology.orchestra.run.smithi172.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-07-06T02:20:22.101 INFO:teuthology.orchestra.run.smithi172.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-06T02:20:22.134 INFO:teuthology.orchestra.run.smithi172.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.1 MB/s | 72 kB 00:00 2023-07-06T02:20:22.176 INFO:teuthology.orchestra.run.smithi172.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-06T02:20:22.218 INFO:teuthology.orchestra.run.smithi172.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2023-07-06T02:20:22.244 INFO:teuthology.orchestra.run.smithi172.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 895 kB/s | 60 kB 00:00 2023-07-06T02:20:22.269 INFO:teuthology.orchestra.run.smithi172.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-06T02:20:22.294 INFO:teuthology.orchestra.run.smithi172.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-06T02:20:22.319 INFO:teuthology.orchestra.run.smithi172.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-07-06T02:20:22.353 INFO:teuthology.orchestra.run.smithi172.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-06T02:20:22.378 INFO:teuthology.orchestra.run.smithi172.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-06T02:20:22.395 INFO:teuthology.orchestra.run.smithi172.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-07-06T02:20:22.437 INFO:teuthology.orchestra.run.smithi172.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-06T02:20:22.479 INFO:teuthology.orchestra.run.smithi172.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-06T02:20:22.530 INFO:teuthology.orchestra.run.smithi172.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-06T02:20:22.571 INFO:teuthology.orchestra.run.smithi172.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-06T02:20:22.605 INFO:teuthology.orchestra.run.smithi172.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-07-06T02:20:22.664 INFO:teuthology.orchestra.run.smithi172.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-07-06T02:20:22.714 INFO:teuthology.orchestra.run.smithi172.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-06T02:20:22.741 INFO:teuthology.orchestra.run.smithi089.stdout:(131/131): librados2-18.0.0-4742.g79365bb3.el8. 887 kB/s | 3.7 MB 00:04 2023-07-06T02:20:22.744 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-07-06T02:20:22.745 INFO:teuthology.orchestra.run.smithi089.stdout:Total 4.9 MB/s | 205 MB 00:41 2023-07-06T02:20:22.756 INFO:teuthology.orchestra.run.smithi172.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-06T02:20:22.798 INFO:teuthology.orchestra.run.smithi172.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-06T02:20:22.932 INFO:teuthology.orchestra.run.smithi172.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 25 MB/s | 14 MB 00:00 2023-07-06T02:20:23.357 INFO:teuthology.orchestra.run.smithi172.stdout:(107/131): ceph-test-18.0.0-4742.g79365bb3.el8. 2.1 MB/s | 48 MB 00:22 2023-07-06T02:20:23.399 INFO:teuthology.orchestra.run.smithi172.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 1.2 MB/s | 713 kB 00:00 2023-07-06T02:20:23.424 INFO:teuthology.orchestra.run.smithi172.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 397 kB/s | 27 kB 00:00 2023-07-06T02:20:23.458 INFO:teuthology.orchestra.run.smithi172.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-07-06T02:20:23.475 INFO:teuthology.orchestra.run.smithi172.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 875 kB/s | 44 kB 00:00 2023-07-06T02:20:23.525 INFO:teuthology.orchestra.run.smithi172.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-07-06T02:20:23.567 INFO:teuthology.orchestra.run.smithi172.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-06T02:20:23.592 INFO:teuthology.orchestra.run.smithi172.stdout:(114/131): uuid-1.6.2-43.el8.x86_64.rpm 2.5 MB/s | 64 kB 00:00 2023-07-06T02:20:23.593 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-07-06T02:20:23.659 INFO:teuthology.orchestra.run.smithi172.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2023-07-06T02:20:23.712 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-07-06T02:20:23.712 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-07-06T02:20:23.726 INFO:teuthology.orchestra.run.smithi172.stdout:(116/131): python3-babel-2.5.1-7.el8.noarch.rpm 6.0 MB/s | 4.8 MB 00:00 2023-07-06T02:20:23.743 INFO:teuthology.orchestra.run.smithi172.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 239 kB/s | 20 kB 00:00 2023-07-06T02:20:23.768 INFO:teuthology.orchestra.run.smithi172.stdout:(118/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 719 kB/s | 126 kB 00:00 2023-07-06T02:20:23.794 INFO:teuthology.orchestra.run.smithi172.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.5 MB/s | 103 kB 00:00 2023-07-06T02:20:23.827 INFO:teuthology.orchestra.run.smithi172.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2023-07-06T02:20:23.853 INFO:teuthology.orchestra.run.smithi172.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-07-06T02:20:23.895 INFO:teuthology.orchestra.run.smithi172.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-06T02:20:23.937 INFO:teuthology.orchestra.run.smithi172.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-07-06T02:20:24.028 INFO:teuthology.orchestra.run.smithi172.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 19 MB/s | 4.9 MB 00:00 2023-07-06T02:20:24.095 INFO:teuthology.orchestra.run.smithi172.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-06T02:20:24.167 INFO:teuthology.orchestra.run.smithi172.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.7 MB/s | 808 kB 00:00 2023-07-06T02:20:24.221 INFO:teuthology.orchestra.run.smithi172.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-06T02:20:24.305 INFO:teuthology.orchestra.run.smithi172.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-07-06T02:20:24.355 INFO:teuthology.orchestra.run.smithi172.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-06T02:20:24.825 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-07-06T02:20:24.829 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-07-06T02:20:26.913 INFO:teuthology.orchestra.run.smithi172.stdout:(130/131): librados2-18.0.0-4742.g79365bb3.el8. 1.3 MB/s | 3.7 MB 00:02 2023-07-06T02:20:29.104 INFO:teuthology.orchestra.run.smithi172.stdout:(131/131): librbd1-18.0.0-4742.g79365bb3.el8.x8 747 kB/s | 3.6 MB 00:04 2023-07-06T02:20:29.107 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2023-07-06T02:20:29.108 INFO:teuthology.orchestra.run.smithi172.stdout:Total 5.2 MB/s | 205 MB 00:39 2023-07-06T02:20:29.606 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-07-06T02:20:29.763 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T02:20:29.914 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2023-07-06T02:20:29.954 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T02:20:30.034 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2023-07-06T02:20:30.034 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2023-07-06T02:20:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T02:20:30.263 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T02:20:30.473 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T02:20:30.853 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T02:20:31.081 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T02:20:31.151 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2023-07-06T02:20:31.155 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2023-07-06T02:20:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T02:20:31.689 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T02:20:31.817 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T02:20:31.997 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T02:20:32.097 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T02:20:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T02:20:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-06T02:20:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T02:20:32.875 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T02:20:33.255 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T02:20:33.501 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T02:20:33.664 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T02:20:33.754 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T02:20:33.922 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T02:20:34.055 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T02:20:34.226 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-06T02:20:34.394 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-06T02:20:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-06T02:20:34.739 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T02:20:34.883 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T02:20:35.030 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T02:20:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T02:20:35.822 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T02:20:36.093 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2023-07-06T02:20:36.276 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T02:20:36.404 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T02:20:36.468 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T02:20:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T02:20:36.629 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T02:20:36.743 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T02:20:36.946 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T02:20:37.283 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T02:20:37.342 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T02:20:37.581 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-06T02:20:37.587 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T02:20:37.761 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-06T02:20:37.980 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T02:20:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T02:20:38.161 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 26/134 2023-07-06T02:20:38.179 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T02:20:38.299 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 27/134 2023-07-06T02:20:38.305 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T02:20:38.471 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T02:20:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 28/134 2023-07-06T02:20:38.568 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T02:20:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T02:20:38.759 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T02:20:38.953 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-06T02:20:39.225 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T02:20:39.432 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T02:20:39.820 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T02:20:40.057 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T02:20:40.237 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T02:20:40.342 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T02:20:40.472 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T02:20:40.512 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T02:20:40.660 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T02:20:40.841 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-06T02:20:41.034 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-06T02:20:41.208 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-06T02:20:41.345 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T02:20:41.493 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T02:20:41.500 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-06T02:20:41.628 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T02:20:41.794 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-06T02:20:42.073 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-06T02:20:42.241 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T02:20:42.300 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-06T02:20:42.428 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T02:20:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T02:20:42.698 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T02:20:42.886 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 36/134 2023-07-06T02:20:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T02:20:43.060 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 37/134 2023-07-06T02:20:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T02:20:43.203 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-06T02:20:43.350 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-06T02:20:43.537 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T02:20:43.605 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T02:20:43.806 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-06T02:20:43.922 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T02:20:44.015 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-06T02:20:44.154 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T02:20:44.172 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-06T02:20:44.234 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T02:20:44.384 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-06T02:20:44.441 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-06T02:20:44.597 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T02:20:44.608 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-06T02:20:44.755 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-06T02:20:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 26/134 2023-07-06T02:20:44.905 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 27/134 2023-07-06T02:20:44.939 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-06T02:20:45.133 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 28/134 2023-07-06T02:20:45.215 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-06T02:20:45.344 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T02:20:45.516 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-06T02:20:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-06T02:20:45.899 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-06T02:20:46.256 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-06T02:20:46.555 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-06T02:20:46.765 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-06T02:20:46.982 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-06T02:20:47.102 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T02:20:47.208 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-06T02:20:47.502 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-06T02:20:47.711 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-06T02:20:47.905 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-06T02:20:48.108 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-06T02:20:48.140 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-06T02:20:48.284 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-06T02:20:48.448 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-06T02:20:48.458 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-06T02:20:48.729 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-06T02:20:48.746 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-06T02:20:48.959 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-06T02:20:48.998 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-06T02:20:49.172 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T02:20:49.224 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-06T02:20:49.379 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T02:20:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-06T02:20:49.558 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 36/134 2023-07-06T02:20:49.636 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T02:20:49.709 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 37/134 2023-07-06T02:20:49.814 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T02:20:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-06T02:20:50.009 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-06T02:20:50.096 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T02:20:50.210 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T02:20:50.279 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T02:20:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T02:20:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T02:20:50.496 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-06T02:20:50.605 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T02:20:50.722 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-06T02:20:50.795 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T02:20:50.868 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T02:20:50.948 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T02:20:51.031 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T02:20:51.149 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-06T02:20:51.205 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T02:20:51.306 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-06T02:20:51.339 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T02:20:51.451 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T02:20:51.480 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-06T02:20:51.658 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T02:20:51.663 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-06T02:20:51.846 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-06T02:20:51.873 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T02:20:51.997 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-06T02:20:52.051 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T02:20:52.185 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-06T02:20:52.291 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T02:20:52.391 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-06T02:20:52.552 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T02:20:52.703 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T02:20:52.770 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-06T02:20:52.903 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T02:20:53.036 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-06T02:20:53.287 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-06T02:20:53.522 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-06T02:20:53.722 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-06T02:20:53.964 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T02:20:54.051 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-06T02:20:54.206 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T02:20:54.293 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-06T02:20:54.436 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T02:20:54.482 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-06T02:20:54.674 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-06T02:20:54.704 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T02:20:54.833 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-06T02:20:54.947 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T02:20:54.996 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-06T02:20:55.144 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T02:20:55.325 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-06T02:20:55.590 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-06T02:20:55.813 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-06T02:20:55.926 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T02:20:56.069 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-06T02:20:56.085 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T02:20:56.156 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T02:20:56.233 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T02:20:56.319 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 93/134 2023-07-06T02:20:56.412 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T02:20:56.670 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T02:20:56.899 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T02:20:57.055 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T02:20:57.219 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T02:20:57.424 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T02:20:57.644 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T02:20:57.835 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T02:20:58.003 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T02:20:58.156 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T02:20:58.370 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T02:20:58.587 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T02:20:58.781 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T02:20:59.015 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T02:20:59.258 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T02:20:59.437 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T02:20:59.660 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T02:20:59.977 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T02:21:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T02:21:00.756 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T02:21:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T02:21:01.020 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T02:21:01.233 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T02:21:01.522 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T02:21:01.525 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T02:21:01.604 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T02:21:01.727 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T02:21:01.794 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T02:21:01.967 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T02:21:02.767 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T02:21:02.983 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T02:21:03.054 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T02:21:03.208 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 93/134 2023-07-06T02:21:06.835 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T02:21:07.049 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T02:21:07.636 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T02:21:07.888 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T02:21:08.084 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T02:21:08.932 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T02:21:21.048 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T02:21:21.048 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2023-07-06T02:21:21.048 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /run 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-07-06T02:21:21.049 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:21:21.390 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T02:21:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T02:21:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T02:21:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T02:21:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T02:21:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T02:21:24.165 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T02:21:24.181 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T02:21:25.872 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T02:21:26.053 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T02:21:26.179 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 102/134 2023-07-06T02:21:26.353 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 103/134 2023-07-06T02:21:26.548 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T02:21:26.763 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T02:21:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T02:21:27.753 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T02:21:27.753 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /dev 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /run 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2023-07-06T02:21:27.754 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /dev 2023-07-06T02:21:27.755 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:21:28.042 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T02:21:28.079 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T02:21:28.222 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T02:21:28.312 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T02:21:28.654 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 107/134 2023-07-06T02:21:28.834 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T02:21:28.869 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T02:21:29.039 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T02:21:29.050 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T02:21:29.657 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T02:21:29.810 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T02:21:30.027 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T02:21:30.162 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T02:21:30.332 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 110/134 2023-07-06T02:21:30.514 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T02:21:30.548 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T02:21:30.617 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T02:21:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T02:21:30.731 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T02:21:30.795 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T02:21:30.811 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T02:21:30.872 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T02:21:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T02:21:31.372 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T02:21:32.409 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T02:21:32.611 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T02:21:32.717 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 102/134 2023-07-06T02:21:32.892 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 103/134 2023-07-06T02:21:33.044 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T02:21:33.259 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T02:21:33.525 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T02:21:34.488 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T02:21:34.644 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T02:21:34.735 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T02:21:35.095 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 107/134 2023-07-06T02:21:35.267 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T02:21:35.301 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T02:21:35.469 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T02:21:35.479 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T02:21:35.899 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T02:21:36.773 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 110/134 2023-07-06T02:21:36.774 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 115/134 2023-07-06T02:21:36.976 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T02:21:37.009 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T02:21:37.094 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-06T02:21:37.152 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T02:21:37.193 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T02:21:37.266 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 117/134 2023-07-06T02:21:37.377 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T02:21:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-06T02:21:37.654 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T02:21:37.723 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T02:21:37.845 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T02:21:37.869 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T02:21:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-06T02:21:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 121/134 2023-07-06T02:21:38.372 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T02:21:40.516 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T02:21:40.746 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T02:21:40.765 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T02:21:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 124/134 2023-07-06T02:21:41.170 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 125/134 2023-07-06T02:21:41.426 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 126/134 2023-07-06T02:21:41.759 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T02:21:41.945 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T02:21:42.028 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T02:21:42.237 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T02:21:42.249 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T02:21:42.395 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T02:21:42.403 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T02:21:42.503 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T02:21:42.531 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-06T02:21:42.531 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T02:21:42.731 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T02:21:42.732 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T02:21:42.959 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T02:21:43.169 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T02:21:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 115/134 2023-07-06T02:21:43.669 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T02:21:43.669 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/134 2023-07-06T02:21:43.669 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/134 2023-07-06T02:21:43.669 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/134 2023-07-06T02:21:43.669 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/134 2023-07-06T02:21:43.670 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/134 2023-07-06T02:21:43.670 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T02:21:43.670 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T02:21:43.670 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T02:21:43.670 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T02:21:43.670 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/134 2023-07-06T02:21:43.670 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/134 2023-07-06T02:21:43.671 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/134 2023-07-06T02:21:43.671 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/134 2023-07-06T02:21:43.671 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/134 2023-07-06T02:21:43.671 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/134 2023-07-06T02:21:43.671 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/134 2023-07-06T02:21:43.671 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/134 2023-07-06T02:21:43.679 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/134 2023-07-06T02:21:43.679 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/134 2023-07-06T02:21:43.680 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/134 2023-07-06T02:21:43.680 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/134 2023-07-06T02:21:43.680 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/134 2023-07-06T02:21:43.680 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/134 2023-07-06T02:21:43.680 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/134 2023-07-06T02:21:43.680 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/134 2023-07-06T02:21:43.680 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/134 2023-07-06T02:21:43.681 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/134 2023-07-06T02:21:43.682 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/134 2023-07-06T02:21:43.682 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/134 2023-07-06T02:21:43.682 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/134 2023-07-06T02:21:43.682 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/134 2023-07-06T02:21:43.682 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T02:21:43.682 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T02:21:43.682 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T02:21:43.683 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T02:21:43.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T02:21:43.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T02:21:43.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T02:21:43.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T02:21:43.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T02:21:43.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T02:21:43.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T02:21:43.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T02:21:43.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T02:21:43.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T02:21:43.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T02:21:43.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T02:21:43.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T02:21:43.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T02:21:43.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T02:21:43.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-06T02:21:43.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-06T02:21:43.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-06T02:21:43.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-06T02:21:43.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-06T02:21:43.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-06T02:21:43.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-06T02:21:43.688 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-06T02:21:43.689 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-06T02:21:43.689 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-06T02:21:43.689 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-06T02:21:43.689 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-06T02:21:43.689 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-06T02:21:43.689 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-06T02:21:43.689 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-06T02:21:43.690 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-06T02:21:43.691 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-06T02:21:43.691 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-06T02:21:43.691 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-06T02:21:43.691 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-06T02:21:43.691 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-06T02:21:43.691 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-06T02:21:43.691 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-06T02:21:43.692 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-06T02:21:43.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-06T02:21:43.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-06T02:21:43.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-06T02:21:43.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-06T02:21:43.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-06T02:21:43.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-06T02:21:43.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-06T02:21:43.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-06T02:21:43.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-06T02:21:43.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-06T02:21:43.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-06T02:21:43.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-06T02:21:43.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/134 2023-07-06T02:21:43.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T02:21:43.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/134 2023-07-06T02:21:43.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T02:21:43.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T02:21:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-06T02:21:43.961 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 117/134 2023-07-06T02:21:44.243 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-06T02:21:44.409 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T02:21:44.600 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T02:21:44.838 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-06T02:21:45.022 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 121/134 2023-07-06T02:21:45.168 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T02:21:46.120 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T02:21:46.121 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:17 ago on Thu 06 Jul 2023 02:19:29 AM UTC. 2023-07-06T02:21:47.292 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T02:21:47.584 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T02:21:47.601 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T02:21:47.776 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 124/134 2023-07-06T02:21:48.024 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 125/134 2023-07-06T02:21:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 126/134 2023-07-06T02:21:48.662 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T02:21:48.833 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T02:21:48.915 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T02:21:48.939 INFO:teuthology.orchestra.run.smithi089.stdout:Installed products updated. 2023-07-06T02:21:49.100 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T02:21:49.112 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T02:21:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:21:49.167 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-07-06T02:21:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T02:21:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.167 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-07-06T02:21:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T02:21:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T02:21:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T02:21:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T02:21:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: jq-1.6-3.el8.x86_64 2023-07-06T02:21:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T02:21:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T02:21:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T02:21:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T02:21:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T02:21:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T02:21:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T02:21:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T02:21:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T02:21:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T02:21:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T02:21:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T02:21:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T02:21:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T02:21:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T02:21:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T02:21:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T02:21:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T02:21:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T02:21:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T02:21:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T02:21:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T02:21:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T02:21:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T02:21:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T02:21:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T02:21:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T02:21:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T02:21:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T02:21:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T02:21:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T02:21:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T02:21:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T02:21:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T02:21:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T02:21:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T02:21:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T02:21:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T02:21:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T02:21:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T02:21:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T02:21:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T02:21:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T02:21:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T02:21:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T02:21:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T02:21:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T02:21:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T02:21:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T02:21:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T02:21:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T02:21:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T02:21:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T02:21:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T02:21:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T02:21:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T02:21:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T02:21:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T02:21:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T02:21:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T02:21:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T02:21:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T02:21:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T02:21:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T02:21:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T02:21:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T02:21:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T02:21:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:21:49.189 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-07-06T02:21:49.292 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T02:21:49.299 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T02:21:49.382 DEBUG:teuthology.parallel:result is None 2023-07-06T02:21:49.435 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-06T02:21:49.435 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T02:21:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T02:21:49.645 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T02:21:49.905 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T02:21:50.090 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T02:21:50.595 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T02:21:50.596 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/134 2023-07-06T02:21:50.596 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/134 2023-07-06T02:21:50.596 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/134 2023-07-06T02:21:50.596 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/134 2023-07-06T02:21:50.596 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/134 2023-07-06T02:21:50.597 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T02:21:50.597 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T02:21:50.597 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T02:21:50.597 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T02:21:50.597 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/134 2023-07-06T02:21:50.597 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/134 2023-07-06T02:21:50.597 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/134 2023-07-06T02:21:50.598 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/134 2023-07-06T02:21:50.599 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/134 2023-07-06T02:21:50.600 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/134 2023-07-06T02:21:50.600 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/134 2023-07-06T02:21:50.606 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/134 2023-07-06T02:21:50.607 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/134 2023-07-06T02:21:50.607 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/134 2023-07-06T02:21:50.607 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/134 2023-07-06T02:21:50.607 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/134 2023-07-06T02:21:50.607 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/134 2023-07-06T02:21:50.607 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/134 2023-07-06T02:21:50.607 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/134 2023-07-06T02:21:50.608 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T02:21:50.608 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T02:21:50.608 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T02:21:50.608 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T02:21:50.608 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T02:21:50.608 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T02:21:50.608 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T02:21:50.609 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T02:21:50.610 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T02:21:50.610 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T02:21:50.610 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T02:21:50.610 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T02:21:50.610 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T02:21:50.611 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T02:21:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T02:21:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T02:21:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T02:21:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T02:21:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T02:21:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-06T02:21:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-06T02:21:50.613 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-06T02:21:50.614 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-06T02:21:50.615 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-06T02:21:50.616 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-06T02:21:50.617 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-06T02:21:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-06T02:21:50.619 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/134 2023-07-06T02:21:50.620 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T02:21:50.620 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/134 2023-07-06T02:21:50.620 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T02:21:50.620 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T02:21:53.076 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T02:21:53.077 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:02:15 ago on Thu 06 Jul 2023 02:19:38 AM UTC. 2023-07-06T02:21:55.738 INFO:teuthology.orchestra.run.smithi172.stdout:Installed products updated. 2023-07-06T02:21:56.037 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:21:56.037 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2023-07-06T02:21:56.038 INFO:teuthology.orchestra.run.smithi172.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T02:21:56.038 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.038 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.038 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2023-07-06T02:21:56.038 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T02:21:56.038 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T02:21:56.038 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T02:21:56.039 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.039 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.039 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.039 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.039 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.039 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.040 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.040 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.040 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.040 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.040 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.040 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.040 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.041 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.042 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.042 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T02:21:56.042 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T02:21:56.042 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T02:21:56.042 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T02:21:56.042 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T02:21:56.042 INFO:teuthology.orchestra.run.smithi172.stdout: jq-1.6-3.el8.x86_64 2023-07-06T02:21:56.043 INFO:teuthology.orchestra.run.smithi172.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T02:21:56.043 INFO:teuthology.orchestra.run.smithi172.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T02:21:56.043 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T02:21:56.043 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.043 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.043 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.043 INFO:teuthology.orchestra.run.smithi172.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T02:21:56.044 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T02:21:56.044 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T02:21:56.044 INFO:teuthology.orchestra.run.smithi172.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T02:21:56.044 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T02:21:56.044 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T02:21:56.044 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.045 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.045 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T02:21:56.045 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.045 INFO:teuthology.orchestra.run.smithi172.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T02:21:56.046 INFO:teuthology.orchestra.run.smithi172.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T02:21:56.046 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T02:21:56.046 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T02:21:56.047 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T02:21:56.047 INFO:teuthology.orchestra.run.smithi172.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T02:21:56.047 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T02:21:56.047 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T02:21:56.048 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T02:21:56.048 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T02:21:56.048 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T02:21:56.048 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T02:21:56.049 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T02:21:56.049 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T02:21:56.049 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T02:21:56.049 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T02:21:56.050 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T02:21:56.050 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T02:21:56.050 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T02:21:56.051 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T02:21:56.051 INFO:teuthology.orchestra.run.smithi172.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T02:21:56.051 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T02:21:56.051 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.052 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.052 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.052 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T02:21:56.053 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T02:21:56.053 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T02:21:56.053 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T02:21:56.053 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T02:21:56.054 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T02:21:56.054 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T02:21:56.054 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T02:21:56.054 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T02:21:56.055 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T02:21:56.055 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T02:21:56.055 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T02:21:56.055 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T02:21:56.056 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T02:21:56.056 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T02:21:56.056 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T02:21:56.057 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T02:21:56.057 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T02:21:56.057 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T02:21:56.057 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T02:21:56.057 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T02:21:56.058 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T02:21:56.058 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T02:21:56.058 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T02:21:56.058 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T02:21:56.058 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T02:21:56.058 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T02:21:56.058 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T02:21:56.059 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T02:21:56.060 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T02:21:56.060 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T02:21:56.060 INFO:teuthology.orchestra.run.smithi172.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T02:21:56.060 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T02:21:56.060 INFO:teuthology.orchestra.run.smithi172.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T02:21:56.060 INFO:teuthology.orchestra.run.smithi172.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T02:21:56.061 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T02:21:56.061 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T02:21:56.062 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T02:21:56.062 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T02:21:56.062 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T02:21:56.062 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.063 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.063 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T02:21:56.063 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T02:21:56.064 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T02:21:56.064 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T02:21:56.064 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T02:21:56.064 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T02:21:56.065 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T02:21:56.065 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:21:56.066 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2023-07-06T02:21:56.247 DEBUG:teuthology.parallel:result is None 2023-07-06T02:21:56.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T02:21:56.392 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T02:21:56.416 INFO:teuthology.orchestra.run.smithi089.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T02:21:56.416 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T02:21:56.417 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T02:21:56.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T02:21:56.631 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T02:21:56.654 INFO:teuthology.orchestra.run.smithi172.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T02:21:56.654 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T02:21:56.656 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T02:21:56.657 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-06T02:21:56.658 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:21:56.659 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T02:21:56.699 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:21:56.700 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T02:21:56.742 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-06T02:21:56.743 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:21:56.743 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T02:21:56.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T02:21:56.837 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:21:56.838 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T02:21:56.864 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T02:21:56.931 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-06T02:21:56.931 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:21:56.931 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T02:21:56.957 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T02:21:57.021 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:21:57.022 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T02:21:57.048 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T02:21:57.116 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-06T02:21:57.116 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:21:57.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T02:21:57.143 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T02:21:57.209 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:21:57.210 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T02:21:57.237 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T02:21:57.304 INFO:teuthology.run_tasks:Running task ceph... 2023-07-06T02:21:57.376 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-06T02:21:57.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2023-07-06T02:21:57.378 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /var/log/ceph 2023-07-06T02:21:57.404 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-06T02:21:57.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T02:21:57.444 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T02:21:57.471 INFO:tasks.ceph:Creating extra log directories... 2023-07-06T02:21:57.471 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T02:21:57.513 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T02:21:57.546 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-06T02:21:57.546 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-06T02:21:57.546 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-06T02:21:57.547 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:21:57.547 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T02:21:57.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T02:21:57.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T02:21:57.848 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:21:57.849 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T02:21:57.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T02:21:58.008 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T02:21:58.040 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-06T02:21:58.040 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-06T02:21:58.040 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327840', 'branch': 'wip-rishabh-2023Jul5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7327840', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jul5'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jul5', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jul5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi089.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi172.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b53f236c1077f1d295baafb9e0ca9a4b447019d7', 'timestamp': '2023-07-05_22:21:20', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-06T02:21:58.041 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T02:21:58.043 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T02:21:58.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T02:21:58.100 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T02:21:58.158 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:21:58.159 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T02:21:58.180 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T02:21:58.180 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2023-07-06T02:21:58.237 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T02:21:58.238 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:58.238 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 375153 Links: 1 2023-07-06T02:21:58.238 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:58.238 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:58.238 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-07-06 02:21:43.372954032 +0000 2023-07-06T02:21:58.238 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-07-06 02:14:24.557194337 +0000 2023-07-06T02:21:58.238 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-07-06 02:14:24.557194337 +0000 2023-07-06T02:21:58.239 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-07-06T02:21:58.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T02:21:58.303 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-07-06T02:21:58.304 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-07-06T02:21:58.304 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000227497 s, 2.3 MB/s 2023-07-06T02:21:58.305 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T02:21:58.362 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2023-07-06T02:21:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 379171 Links: 1 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-07-06 02:21:43.373954009 +0000 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-07-06 02:14:25.322176974 +0000 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-07-06 02:14:25.322176974 +0000 2023-07-06T02:21:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-07-06T02:21:58.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T02:21:58.483 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-07-06T02:21:58.484 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-07-06T02:21:58.484 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000213538 s, 2.4 MB/s 2023-07-06T02:21:58.485 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T02:21:58.543 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2023-07-06T02:21:58.600 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T02:21:58.600 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:58.600 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 376527 Links: 1 2023-07-06T02:21:58.601 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:58.601 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:58.601 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-07-06 02:21:43.373954009 +0000 2023-07-06T02:21:58.601 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-07-06 02:14:25.888164127 +0000 2023-07-06T02:21:58.601 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-07-06 02:14:25.888164127 +0000 2023-07-06T02:21:58.601 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-07-06T02:21:58.602 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T02:21:58.666 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-07-06T02:21:58.666 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-07-06T02:21:58.667 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.00020152 s, 2.5 MB/s 2023-07-06T02:21:58.667 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T02:21:58.724 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2023-07-06T02:21:58.781 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T02:21:58.781 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:58.781 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 383408 Links: 1 2023-07-06T02:21:58.781 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:58.782 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:58.782 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-07-06 02:21:43.373954009 +0000 2023-07-06T02:21:58.782 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-07-06 02:14:26.393152665 +0000 2023-07-06T02:21:58.783 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-07-06 02:14:26.393152665 +0000 2023-07-06T02:21:58.783 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-07-06T02:21:58.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T02:21:58.846 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-07-06T02:21:58.847 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-07-06T02:21:58.848 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000218261 s, 2.3 MB/s 2023-07-06T02:21:58.849 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T02:21:58.907 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T02:21:58.907 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:21:58.908 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T02:21:58.924 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T02:21:58.925 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2023-07-06T02:21:58.981 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T02:21:58.981 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:58.982 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 382991 Links: 1 2023-07-06T02:21:58.982 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:58.982 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:58.982 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-07-06 02:21:50.293407278 +0000 2023-07-06T02:21:58.982 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-07-06 02:18:39.406074610 +0000 2023-07-06T02:21:58.982 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-07-06 02:18:39.406074610 +0000 2023-07-06T02:21:58.982 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-07-06T02:21:58.983 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T02:21:59.046 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-07-06T02:21:59.047 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-07-06T02:21:59.047 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000241526 s, 2.1 MB/s 2023-07-06T02:21:59.048 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T02:21:59.106 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2023-07-06T02:21:59.164 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T02:21:59.164 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:59.165 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 380385 Links: 1 2023-07-06T02:21:59.165 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:59.165 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:59.166 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-07-06 02:21:50.293407278 +0000 2023-07-06T02:21:59.166 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-07-06 02:18:39.939061753 +0000 2023-07-06T02:21:59.166 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-07-06 02:18:39.939061753 +0000 2023-07-06T02:21:59.167 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-07-06T02:21:59.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T02:21:59.231 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-07-06T02:21:59.231 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-07-06T02:21:59.231 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000225621 s, 2.3 MB/s 2023-07-06T02:21:59.233 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T02:21:59.292 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2023-07-06T02:21:59.350 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T02:21:59.350 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:59.350 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 380440 Links: 1 2023-07-06T02:21:59.351 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:59.351 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:59.351 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-07-06 02:21:50.293407278 +0000 2023-07-06T02:21:59.351 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-07-06 02:18:40.449049451 +0000 2023-07-06T02:21:59.351 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-07-06 02:18:40.449049451 +0000 2023-07-06T02:21:59.351 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-07-06T02:21:59.352 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T02:21:59.415 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-07-06T02:21:59.416 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-07-06T02:21:59.416 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000202186 s, 2.5 MB/s 2023-07-06T02:21:59.417 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T02:21:59.475 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2023-07-06T02:21:59.532 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T02:21:59.532 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T02:21:59.532 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 382810 Links: 1 2023-07-06T02:21:59.532 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T02:21:59.533 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T02:21:59.533 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-07-06 02:21:50.293407278 +0000 2023-07-06T02:21:59.533 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-07-06 02:18:40.944037511 +0000 2023-07-06T02:21:59.533 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-07-06 02:18:40.944037511 +0000 2023-07-06T02:21:59.533 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-07-06T02:21:59.534 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T02:21:59.598 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-07-06T02:21:59.598 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-07-06T02:21:59.598 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000204451 s, 2.5 MB/s 2023-07-06T02:21:59.599 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T02:21:59.657 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T02:21:59.657 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T02:21:59.657 INFO:tasks.ceph:Generating config... 2023-07-06T02:21:59.658 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-06T02:21:59.659 INFO:tasks.ceph:[client] debug client = 20 2023-07-06T02:21:59.659 INFO:tasks.ceph:[client] debug ms = 1 2023-07-06T02:21:59.659 INFO:tasks.ceph:[client] pid file = 2023-07-06T02:21:59.659 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-06T02:21:59.659 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-06T02:21:59.660 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-07-06T02:21:59.660 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-06T02:21:59.660 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-06T02:21:59.660 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-06T02:21:59.661 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-06T02:21:59.661 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-06T02:21:59.661 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-06T02:21:59.661 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-06T02:21:59.661 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-06T02:21:59.662 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-06T02:21:59.662 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-06T02:21:59.662 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-06T02:21:59.662 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-06T02:21:59.662 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-06T02:21:59.663 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-06T02:21:59.663 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-06T02:21:59.663 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-06T02:21:59.663 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-06T02:21:59.663 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-06T02:21:59.664 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-07-06T02:21:59.664 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-07-06T02:21:59.664 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-06T02:21:59.664 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-06T02:21:59.664 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-07-06T02:21:59.665 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-06T02:21:59.665 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-06T02:21:59.665 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-06T02:21:59.665 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-06T02:21:59.665 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-06T02:21:59.666 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-06T02:21:59.666 INFO:tasks.ceph:Setting up mon.a... 2023-07-06T02:21:59.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-06T02:21:59.719 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.keyring 2023-07-06T02:21:59.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-06T02:21:59.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T02:21:59.793 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.89'), ('mon.b', '172.21.15.172'), ('mon.c', '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]')] 2023-07-06T02:21:59.795 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.89,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-06T02:21:59.796 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:21:59.797 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T02:21:59.849 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.89 --add b 172.21.15.172 --addv c '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:21:59.938 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:21:59.938 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: generated fsid b61ed025-dcde-49f5-a87d-8904066cfa54 2023-07-06T02:21:59.939 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = pacific 2023-07-06T02:21:59.939 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 0 2023-07-06T02:21:59.939 INFO:teuthology.orchestra.run.smithi089.stdout:fsid b61ed025-dcde-49f5-a87d-8904066cfa54 2023-07-06T02:21:59.940 INFO:teuthology.orchestra.run.smithi089.stdout:last_changed 2023-07-06T02:21:59.936694+0000 2023-07-06T02:21:59.940 INFO:teuthology.orchestra.run.smithi089.stdout:created 2023-07-06T02:21:59.936694+0000 2023-07-06T02:21:59.940 INFO:teuthology.orchestra.run.smithi089.stdout:min_mon_release 16 (pacific) 2023-07-06T02:21:59.941 INFO:teuthology.orchestra.run.smithi089.stdout:election_strategy: 1 2023-07-06T02:21:59.941 INFO:teuthology.orchestra.run.smithi089.stdout:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2023-07-06T02:21:59.942 INFO:teuthology.orchestra.run.smithi089.stdout:1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2023-07-06T02:21:59.942 INFO:teuthology.orchestra.run.smithi089.stdout:2: [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] mon.c 2023-07-06T02:21:59.942 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-06T02:21:59.944 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T02:21:59.960 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b61ed025-dcde-49f5-a87d-8904066cfa54... 2023-07-06T02:21:59.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T02:22:00.003 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T02:22:00.047 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2023-07-06T02:22:00.048 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2023-07-06T02:22:00.048 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T02:22:00.048 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2023-07-06T02:22:00.049 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.049 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2023-07-06T02:22:00.049 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.049 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2023-07-06T02:22:00.050 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.050 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2023-07-06T02:22:00.050 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2023-07-06T02:22:00.050 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.051 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2023-07-06T02:22:00.051 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2023-07-06T02:22:00.051 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.051 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T02:22:00.052 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2023-07-06T02:22:00.052 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.052 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T02:22:00.053 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T02:22:00.053 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.053 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2023-07-06T02:22:00.053 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.054 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2023-07-06T02:22:00.054 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2023-07-06T02:22:00.054 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2023-07-06T02:22:00.054 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2023-07-06T02:22:00.054 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2023-07-06T02:22:00.055 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2023-07-06T02:22:00.055 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2023-07-06T02:22:00.055 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T02:22:00.055 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T02:22:00.055 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2023-07-06T02:22:00.055 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.055 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T02:22:00.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.056 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2023-07-06T02:22:00.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.056 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2023-07-06T02:22:00.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.056 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2023-07-06T02:22:00.057 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2023-07-06T02:22:00.057 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2023-07-06T02:22:00.057 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = "172.21.15.89,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2023-07-06T02:22:00.057 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on skipped message = False 2023-07-06T02:22:00.057 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = b61ed025-dcde-49f5-a87d-8904066cfa54 2023-07-06T02:22:00.057 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.057 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 600 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock profile = high_recovery_ops 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2023-07-06T02:22:00.058 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2023-07-06T02:22:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2023-07-06T02:22:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2023-07-06T02:22:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2023-07-06T02:22:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2023-07-06T02:22:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2023-07-06T02:22:00.060 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.060 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2023-07-06T02:22:00.060 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.060 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2023-07-06T02:22:00.060 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.060 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2023-07-06T02:22:00.060 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2023-07-06T02:22:00.061 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 20 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 20 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 10 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2023-07-06T02:22:00.062 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2023-07-06T02:22:00.063 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2023-07-06T02:22:00.063 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-07-06T02:22:00.063 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.063 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2023-07-06T02:22:00.063 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-07-06T02:22:00.063 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2023-07-06T02:22:00.063 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-07-06T02:22:00.064 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-07-06T02:22:00.064 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-07-06T02:22:00.064 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-07-06T02:22:00.064 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2023-07-06T02:22:00.064 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.064 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2023-07-06T02:22:00.064 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2023-07-06T02:22:00.065 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2023-07-06T02:22:00.066 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-07-06T02:22:00.066 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.066 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T02:22:00.066 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T02:22:00.066 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T02:22:00.066 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.067 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T02:22:00.067 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T02:22:00.067 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T02:22:00.067 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.067 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2023-07-06T02:22:00.067 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2023-07-06T02:22:00.067 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.068 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2023-07-06T02:22:00.068 INFO:teuthology.orchestra.run.smithi089.stdout: mon op complaint time = 120 2023-07-06T02:22:00.068 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:00.068 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2023-07-06T02:22:00.068 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2023-07-06T02:22:00.068 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2023-07-06T02:22:00.068 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: client mount timeout = 600 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: debug client = 20 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = "" 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-07-06T02:22:00.069 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-07-06T02:22:00.070 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2023-07-06T02:22:00.070 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.b] 2023-07-06T02:22:00.070 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.c] 2023-07-06T02:22:00.070 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-07-06T02:22:00.070 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c] 2023-07-06T02:22:00.071 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b] 2023-07-06T02:22:00.071 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d] 2023-07-06T02:22:00.071 INFO:teuthology.orchestra.run.smithi089.stdout:[mds] 2023-07-06T02:22:00.071 INFO:teuthology.orchestra.run.smithi089.stdout: debug mds = 20 2023-07-06T02:22:00.071 INFO:teuthology.orchestra.run.smithi089.stdout: debug mds balancer = 20 2023-07-06T02:22:00.072 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-07-06T02:22:00.072 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug frag = True 2023-07-06T02:22:00.072 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug scatterstat = True 2023-07-06T02:22:00.072 INFO:teuthology.orchestra.run.smithi089.stdout: mds op complaint time = 180 2023-07-06T02:22:00.072 INFO:teuthology.orchestra.run.smithi089.stdout: mds verify scatter = True 2023-07-06T02:22:00.072 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-07-06T02:22:00.073 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-07-06T02:22:00.073 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-07-06T02:22:00.075 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2023-07-06T02:22:00.076 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2023-07-06T02:22:00.076 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T02:22:00.076 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2023-07-06T02:22:00.076 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.077 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2023-07-06T02:22:00.077 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.077 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2023-07-06T02:22:00.078 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.078 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2023-07-06T02:22:00.078 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2023-07-06T02:22:00.078 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.079 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2023-07-06T02:22:00.079 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2023-07-06T02:22:00.079 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.080 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T02:22:00.080 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2023-07-06T02:22:00.080 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.081 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T02:22:00.081 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T02:22:00.081 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.081 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2023-07-06T02:22:00.082 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.082 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2023-07-06T02:22:00.082 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2023-07-06T02:22:00.082 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2023-07-06T02:22:00.082 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2023-07-06T02:22:00.083 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2023-07-06T02:22:00.083 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2023-07-06T02:22:00.083 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2023-07-06T02:22:00.083 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T02:22:00.084 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T02:22:00.084 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2023-07-06T02:22:00.084 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.084 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2023-07-06T02:22:00.085 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2023-07-06T02:22:00.086 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.89,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2023-07-06T02:22:00.086 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2023-07-06T02:22:00.086 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = b61ed025-dcde-49f5-a87d-8904066cfa54 2023-07-06T02:22:00.086 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.086 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2023-07-06T02:22:00.086 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2023-07-06T02:22:00.086 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.087 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2023-07-06T02:22:00.087 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2023-07-06T02:22:00.087 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2023-07-06T02:22:00.087 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.087 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2023-07-06T02:22:00.087 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2023-07-06T02:22:00.087 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2023-07-06T02:22:00.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.089 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2023-07-06T02:22:00.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.089 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2023-07-06T02:22:00.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.089 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2023-07-06T02:22:00.089 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2023-07-06T02:22:00.090 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2023-07-06T02:22:00.091 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2023-07-06T02:22:00.092 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.092 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2023-07-06T02:22:00.092 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-07-06T02:22:00.092 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2023-07-06T02:22:00.092 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2023-07-06T02:22:00.092 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2023-07-06T02:22:00.092 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2023-07-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2023-07-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2023-07-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2023-07-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-07-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2023-07-06T02:22:00.093 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2023-07-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2023-07-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2023-07-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2023-07-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2023-07-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2023-07-06T02:22:00.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2023-07-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2023-07-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.095 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T02:22:00.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T02:22:00.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T02:22:00.096 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.096 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2023-07-06T02:22:00.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2023-07-06T02:22:00.096 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T02:22:00.097 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T02:22:00.098 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2023-07-06T02:22:00.098 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2023-07-06T02:22:00.098 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-07-06T02:22:00.098 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2023-07-06T02:22:00.098 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2023-07-06T02:22:00.098 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2023-07-06T02:22:00.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2023-07-06T02:22:00.099 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2023-07-06T02:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2023-07-06T02:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-07-06T02:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2023-07-06T02:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2023-07-06T02:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2023-07-06T02:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2023-07-06T02:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2023-07-06T02:22:00.101 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2023-07-06T02:22:00.101 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2023-07-06T02:22:00.101 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-06T02:22:00.101 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-06T02:22:00.173 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-06T02:22:00.174 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:00.174 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-06T02:22:00.190 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:00.191 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-06T02:22:00.246 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:22:00.247 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:00.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T02:22:00.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T02:22:00.323 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:00.324 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:00.380 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:22:00.381 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:00.381 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T02:22:00.382 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T02:22:00.420 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:00.420 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:00.476 INFO:tasks.ceph:Setting up mon nodes... 2023-07-06T02:22:00.477 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-06T02:22:00.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T02:22:00.532 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T02:22:00.534 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T02:22:00.600 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T02:22:00.603 INFO:tasks.ceph:Setting up mds nodes... 2023-07-06T02:22:00.603 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T02:22:00.657 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T02:22:00.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-06T02:22:00.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T02:22:00.781 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T02:22:00.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-06T02:22:00.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T02:22:00.863 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T02:22:00.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-06T02:22:00.893 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T02:22:00.989 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T02:22:00.991 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-06T02:22:01.019 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-06T02:22:01.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-06T02:22:01.063 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-06T02:22:01.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-06T02:22:01.159 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-06T02:22:01.173 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-06T02:22:01.216 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-06T02:22:01.230 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-06T02:22:01.314 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-06T02:22:01.328 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-06T02:22:01.328 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T02:22:01.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-06T02:22:01.354 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:01.354 INFO:tasks.ceph:role: osd.0 2023-07-06T02:22:01.355 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:22:01.355 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T02:22:01.430 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:01.430 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:01.431 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:01.431 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:01.431 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:01.855 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-07-06T02:22:01.857 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-07-06T02:22:01.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-06T02:22:01.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-06T02:22:02.025 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-06T02:22:02.092 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:02.092 INFO:tasks.ceph:role: osd.1 2023-07-06T02:22:02.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:22:02.092 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T02:22:02.164 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:02.164 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:02.165 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:02.165 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:02.165 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:02.165 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:02.165 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:02.165 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:02.166 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:02.166 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:02.582 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-07-06T02:22:02.584 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-07-06T02:22:02.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-06T02:22:02.669 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-06T02:22:02.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-06T02:22:02.765 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:02.765 INFO:tasks.ceph:role: osd.2 2023-07-06T02:22:02.766 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:22:02.766 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T02:22:02.839 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:02.840 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:02.840 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:02.841 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:02.841 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:02.841 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:02.841 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:02.842 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:02.842 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:02.842 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:03.277 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-07-06T02:22:03.279 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-07-06T02:22:03.279 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-06T02:22:03.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-06T02:22:03.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-06T02:22:03.485 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:03.486 INFO:tasks.ceph:role: osd.3 2023-07-06T02:22:03.486 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:22:03.486 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T02:22:03.560 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:03.561 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:03.561 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:03.561 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:03.562 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:04.002 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-07-06T02:22:04.004 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-07-06T02:22:04.004 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-06T02:22:04.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-06T02:22:04.209 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:04.403+0000 7f61e0a36640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-06T02:22:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:04.403+0000 7f61e0a36640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-06T02:22:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:04.403+0000 7f61e0a36640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T02:22:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:04.403+0000 7f61e0a36640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T02:22:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:04.403+0000 7f61e0a36640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T02:22:04.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:04.408+0000 7f61e0a36640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-06T02:22:07.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-06T02:22:07.324 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:07.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:07.407+0000 7f09a8c20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-06T02:22:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:07.408+0000 7f09a8c20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-06T02:22:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:07.408+0000 7f09a8c20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T02:22:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:07.408+0000 7f09a8c20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T02:22:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:07.408+0000 7f09a8c20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T02:22:07.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:07.408+0000 7f09a8c20640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-06T02:22:10.307 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-06T02:22:10.334 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:10.419+0000 7f51f70e4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-06T02:22:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:10.420+0000 7f51f70e4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-06T02:22:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:10.420+0000 7f51f70e4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T02:22:10.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:10.420+0000 7f51f70e4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T02:22:10.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:10.420+0000 7f51f70e4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T02:22:10.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:10.420+0000 7f51f70e4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-06T02:22:13.294 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-06T02:22:13.326 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:13.416+0000 7f9df58e2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-06T02:22:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:13.416+0000 7f9df58e2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-06T02:22:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:13.416+0000 7f9df58e2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T02:22:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:13.416+0000 7f9df58e2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T02:22:13.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:13.416+0000 7f9df58e2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T02:22:13.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:13.416+0000 7f9df58e2640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-06T02:22:16.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-06T02:22:16.316 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-06T02:22:16.346 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:16.346 INFO:tasks.ceph:role: osd.4 2023-07-06T02:22:16.347 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:22:16.347 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T02:22:16.423 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:16.424 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:16.424 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:16.424 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:16.424 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:16.424 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:16.424 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:16.424 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:16.425 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:16.425 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:16.856 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-07-06T02:22:16.859 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-07-06T02:22:16.859 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-06T02:22:17.045 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-06T02:22:17.082 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-06T02:22:17.151 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:17.152 INFO:tasks.ceph:role: osd.5 2023-07-06T02:22:17.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:22:17.152 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T02:22:17.226 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:17.227 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:17.227 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:17.227 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:17.227 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:17.648 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-07-06T02:22:17.651 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-07-06T02:22:17.651 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-06T02:22:17.759 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-06T02:22:17.839 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-06T02:22:17.866 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:17.866 INFO:tasks.ceph:role: osd.6 2023-07-06T02:22:17.866 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:22:17.867 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T02:22:17.941 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:17.942 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:17.943 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:18.385 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-07-06T02:22:18.387 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-07-06T02:22:18.387 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-06T02:22:18.559 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-06T02:22:18.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-06T02:22:18.657 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T02:22:18.657 INFO:tasks.ceph:role: osd.7 2023-07-06T02:22:18.657 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:22:18.658 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T02:22:18.731 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T02:22:18.732 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T02:22:18.732 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T02:22:18.732 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T02:22:18.732 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T02:22:18.732 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-07-06T02:22:18.733 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T02:22:18.733 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T02:22:18.733 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T02:22:18.733 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T02:22:19.174 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-07-06T02:22:19.175 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-07-06T02:22:19.176 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-06T02:22:19.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-06T02:22:19.387 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:19.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:19.492+0000 7f722e645640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-06T02:22:19.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:19.492+0000 7f722e645640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-06T02:22:19.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:19.492+0000 7f722e645640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T02:22:19.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:19.492+0000 7f722e645640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T02:22:19.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:19.492+0000 7f722e645640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T02:22:19.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:19.492+0000 7f722e645640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-06T02:22:22.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-06T02:22:22.393 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:22.485 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:22.482+0000 7f2770c10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-06T02:22:22.486 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:22.482+0000 7f2770c10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-06T02:22:22.486 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:22.482+0000 7f2770c10640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T02:22:22.486 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:22.482+0000 7f2770c10640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T02:22:22.486 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:22.482+0000 7f2770c10640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T02:22:22.486 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:22.483+0000 7f2770c10640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-06T02:22:25.354 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-06T02:22:25.387 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:25.479 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:25.476+0000 7f35767e0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-06T02:22:25.479 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:25.476+0000 7f35767e0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-06T02:22:25.479 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:25.476+0000 7f35767e0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T02:22:25.479 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:25.476+0000 7f35767e0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T02:22:25.479 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:25.476+0000 7f35767e0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T02:22:25.480 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:25.476+0000 7f35767e0640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-06T02:22:28.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:22:28.045 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:22:28.342 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-06T02:22:28.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:28.462 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:28.459+0000 7fdf50956640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-06T02:22:28.462 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:28.459+0000 7fdf50956640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-06T02:22:28.463 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:28.459+0000 7fdf50956640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T02:22:28.463 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:28.459+0000 7fdf50956640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T02:22:28.463 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:28.459+0000 7fdf50956640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T02:22:28.463 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:28.460+0000 7fdf50956640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-06T02:22:31.320 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-06T02:22:31.351 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-06T02:22:31.352 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:31.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-06T02:22:31.384 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:31.384 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-06T02:22:31.454 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:31.455 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-06T02:22:31.524 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:31.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-06T02:22:31.594 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:31.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-06T02:22:31.661 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:31.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-06T02:22:31.728 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:31.729 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-06T02:22:31.795 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:31.796 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-06T02:22:31.827 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:31.828 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-06T02:22:31.897 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:31.898 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-06T02:22:31.966 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:31.967 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-06T02:22:32.034 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:32.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-06T02:22:32.101 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:32.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-06T02:22:32.168 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:32.169 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-06T02:22:32.235 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:32.236 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-06T02:22:32.255 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:32.255 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-06T02:22:32.313 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:32.313 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-06T02:22:32.330 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:32.331 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-06T02:22:32.388 INFO:tasks.ceph:Adding keys to all mons... 2023-07-06T02:22:32.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T02:22:32.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T02:22:32.430 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.y] 2023-07-06T02:22:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBIJaZkEdagHxAA9/2TvHUk+ijKGs7jqWs4Uw== 2023-07-06T02:22:32.431 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-07-06T02:22:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBIJaZkxc0eJxAArsU7ZBxo424bxLsp4XDQzg== 2023-07-06T02:22:32.431 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c] 2023-07-06T02:22:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBIJaZk4Kp8LhAA7ZzUGSx1VH4/nIry1no9mw== 2023-07-06T02:22:32.431 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.0] 2023-07-06T02:22:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBMJaZkNQohGBAAZP652zOWUgQp13HnR6+F8A== 2023-07-06T02:22:32.432 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.1] 2023-07-06T02:22:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBPJaZksNJiGBAARGc+H97BjsQQDZN7ecj0hg== 2023-07-06T02:22:32.432 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.2] 2023-07-06T02:22:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBSJaZkCekLGRAAr8LzcL/AGb2ZoNpXqM2eWA== 2023-07-06T02:22:32.432 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.3] 2023-07-06T02:22:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBVJaZkCCbTGBAAx7qRQFCR8YGiZVPXE1zI2g== 2023-07-06T02:22:32.433 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.x] 2023-07-06T02:22:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBIJaZk+NmwIxAA8izORTN1WAcvae0VX/dzlg== 2023-07-06T02:22:32.433 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b] 2023-07-06T02:22:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBIJaZkdkpjMxAAgucXDyMKlD5CNNhUd2FVsw== 2023-07-06T02:22:32.433 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d] 2023-07-06T02:22:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBIJaZkJZnhOhAArhNFsiedGlQACplx1ksQlA== 2023-07-06T02:22:32.433 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.4] 2023-07-06T02:22:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBbJaZks3RsHRAAMWA/q3s6vfs4F5bDUN0MlA== 2023-07-06T02:22:32.434 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.5] 2023-07-06T02:22:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBeJaZkE8vXHBAAXrYe0Q68x+YFaHH2u1byjg== 2023-07-06T02:22:32.434 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.6] 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBhJaZkpgJ1HBAA/FE2+xljcEhieO2g7VmuTg== 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.7] 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBkJaZkdUl6GxAAz4jyZZYYmRikOz52+h97DA== 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBJJaZkfY62AxAAWC7BJI+7fFi+ew8E4NJVSA== 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout:[client.2] 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBJJaZkf3VxCRAA8VDNglGJ+NZaFkDTckVb+Q== 2023-07-06T02:22:32.435 INFO:teuthology.orchestra.run.smithi089.stdout:[client.1] 2023-07-06T02:22:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBJJaZkqSXYDBAAadl28XEDPg/BP/C0riEa6w== 2023-07-06T02:22:32.436 INFO:teuthology.orchestra.run.smithi089.stdout:[client.3] 2023-07-06T02:22:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBJJaZkGdOoEhAAaC9UVh7c1lNsWOPavhp0Xw== 2023-07-06T02:22:32.452 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.y] 2023-07-06T02:22:32.453 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBIJaZkEdagHxAA9/2TvHUk+ijKGs7jqWs4Uw== 2023-07-06T02:22:32.453 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2023-07-06T02:22:32.453 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBIJaZkxc0eJxAArsU7ZBxo424bxLsp4XDQzg== 2023-07-06T02:22:32.453 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2023-07-06T02:22:32.453 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBIJaZk4Kp8LhAA7ZzUGSx1VH4/nIry1no9mw== 2023-07-06T02:22:32.453 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.0] 2023-07-06T02:22:32.454 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMJaZkNQohGBAAZP652zOWUgQp13HnR6+F8A== 2023-07-06T02:22:32.454 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.1] 2023-07-06T02:22:32.454 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBPJaZksNJiGBAARGc+H97BjsQQDZN7ecj0hg== 2023-07-06T02:22:32.454 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.2] 2023-07-06T02:22:32.454 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBSJaZkCekLGRAAr8LzcL/AGb2ZoNpXqM2eWA== 2023-07-06T02:22:32.454 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.3] 2023-07-06T02:22:32.454 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVJaZkCCbTGBAAx7qRQFCR8YGiZVPXE1zI2g== 2023-07-06T02:22:32.455 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.x] 2023-07-06T02:22:32.455 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBIJaZk+NmwIxAA8izORTN1WAcvae0VX/dzlg== 2023-07-06T02:22:32.455 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2023-07-06T02:22:32.455 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBIJaZkdkpjMxAAgucXDyMKlD5CNNhUd2FVsw== 2023-07-06T02:22:32.456 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2023-07-06T02:22:32.456 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBIJaZkJZnhOhAArhNFsiedGlQACplx1ksQlA== 2023-07-06T02:22:32.456 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.4] 2023-07-06T02:22:32.456 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBbJaZks3RsHRAAMWA/q3s6vfs4F5bDUN0MlA== 2023-07-06T02:22:32.456 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.5] 2023-07-06T02:22:32.456 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBeJaZkE8vXHBAAXrYe0Q68x+YFaHH2u1byjg== 2023-07-06T02:22:32.457 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.6] 2023-07-06T02:22:32.457 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBhJaZkpgJ1HBAA/FE2+xljcEhieO2g7VmuTg== 2023-07-06T02:22:32.457 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.7] 2023-07-06T02:22:32.457 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBkJaZkdUl6GxAAz4jyZZYYmRikOz52+h97DA== 2023-07-06T02:22:32.458 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2023-07-06T02:22:32.458 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBJJaZkfY62AxAAWC7BJI+7fFi+ew8E4NJVSA== 2023-07-06T02:22:32.458 INFO:teuthology.orchestra.run.smithi172.stdout:[client.2] 2023-07-06T02:22:32.458 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBJJaZkf3VxCRAA8VDNglGJ+NZaFkDTckVb+Q== 2023-07-06T02:22:32.458 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2023-07-06T02:22:32.458 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBJJaZkqSXYDBAAadl28XEDPg/BP/C0riEa6w== 2023-07-06T02:22:32.459 INFO:teuthology.orchestra.run.smithi172.stdout:[client.3] 2023-07-06T02:22:32.459 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBJJaZkGdOoEhAAaC9UVh7c1lNsWOPavhp0Xw== 2023-07-06T02:22:32.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T02:22:32.480 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T02:22:32.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:32.555 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:32.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:32.613 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:32.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.671 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.835 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:32.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T02:22:32.893 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T02:22:32.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:32.949 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:33.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:33.008 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T02:22:33.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.120 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.122 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.175 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.233 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T02:22:33.285 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.288 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.346 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.466 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T02:22:33.522 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-06T02:22:33.522 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-06T02:22:33.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T02:22:33.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-06T02:22:33.674 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-06T02:22:33.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T02:22:33.799 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-06T02:22:33.825 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-06T02:22:33.892 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T02:22:33.991 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-06T02:22:34.017 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:34.020 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T02:22:34.074 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-06T02:22:34.074 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-06T02:22:34.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-06T02:22:34.077 INFO:tasks.ceph.mon.a:Started 2023-07-06T02:22:34.077 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-06T02:22:34.077 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-06T02:22:34.115 INFO:tasks.ceph.mon.b:Started 2023-07-06T02:22:34.116 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-06T02:22:34.116 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-06T02:22:34.118 INFO:tasks.ceph.mon.c:Started 2023-07-06T02:22:34.118 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-06T02:22:34.119 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-06T02:22:34.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-06T02:22:34.121 INFO:tasks.ceph.mgr.y:Started 2023-07-06T02:22:34.122 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-06T02:22:34.122 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-06T02:22:34.125 INFO:tasks.ceph.mgr.x:Started 2023-07-06T02:22:34.125 INFO:tasks.ceph:Setting crush tunables to default 2023-07-06T02:22:34.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd crush tunables default 2023-07-06T02:22:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.247+0000 7ffb31bfa700 1 Processor -- start 2023-07-06T02:22:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.247+0000 7ffb31bfa700 1 -- start start 2023-07-06T02:22:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb31bfa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb2c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb31bfa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c0cd640 con 0x7ffb2c0d1de0 2023-07-06T02:22:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb31bfa700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c0d1300 con 0x7ffb2c15c600 2023-07-06T02:22:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb31bfa700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0d15e0 con 0x7ffb2c06c3b0 2023-07-06T02:22:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb2affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ffb2c0d1de0 0x7ffb2c06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56210/0 (socket says 172.21.15.89:56210) 2023-07-06T02:22:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb2affd700 1 -- 172.21.15.89:0/198751964 learned_addr learned my addr 172.21.15.89:0/198751964 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb2c0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb2bfff700 1 -- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 msgr2=0x7ffb2c0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-07-06T02:22:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb2bfff700 1 -- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 msgr2=0x7ffb2c0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-06T02:22:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb2c0b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T02:22:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.248+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb2c0b5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T02:22:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.434+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919873674 0 0) 0x7ffb2c0cd640 con 0x7ffb2c0d1de0 2023-07-06T02:22:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.434+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb140035b0 con 0x7ffb2c0d1de0 2023-07-06T02:22:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576283138 0 0) 0x7ffb140035b0 con 0x7ffb2c0d1de0 2023-07-06T02:22:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 msgr2=0x7ffb2c0b5ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:22:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb2c0b5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 >> v1:172.21.15.172:6789/0 conn(0x7ffb2c15c600 legacy=0x7ffb2c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:34.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0d18c0 con 0x7ffb2c0d1de0 2023-07-06T02:22:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7ffb1c001670 con 0x7ffb2c0d1de0 2023-07-06T02:22:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 >> v1:172.21.15.89:6789/0 conn(0x7ffb2c0d1de0 legacy=0x7ffb2c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb14003a30 0x7ffb14003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14004850 0x7ffb14004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb14005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0d15e0 con 0x7ffb14003a30 2023-07-06T02:22:34.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1400bbd0 con 0x7ffb14004850 2023-07-06T02:22:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1400bd90 con 0x7ffb2c06c3b0 2023-07-06T02:22:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb14005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2affd700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14004850 0x7ffb14004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.435+0000 7ffb2b7fe700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb14003a30 0x7ffb14003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2b7fe700 1 -- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 msgr2=0x7ffb14005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2b7fe700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb14005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2b7fe700 1 -- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14004850 msgr2=0x7ffb14004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2b7fe700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14004850 0x7ffb14004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2b7fe700 1 -- 172.21.15.89:0/198751964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140035b0 con 0x7ffb14003a30 2023-07-06T02:22:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c06c3b0 0x7ffb14005240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2b7fe700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb14003a30 0x7ffb14003e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb200017c0 tx=0x7ffb14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da6992f5290526 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200052d0 con 0x7ffb14003a30 2023-07-06T02:22:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20005430 con 0x7ffb14003a30 2023-07-06T02:22:34.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.436+0000 7ffb2a7fc700 1 -- 172.21.15.89:0/198751964 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200038c0 con 0x7ffb14003a30 2023-07-06T02:22:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.437+0000 7ffb31bfa700 1 -- 172.21.15.89:0/198751964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb14003a30 msgr2=0x7ffb14003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.437+0000 7ffb31bfa700 1 --2- 172.21.15.89:0/198751964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb14003a30 0x7ffb14003e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb200017c0 tx=0x7ffb14005880 comp rx=0 tx=0).stop 2023-07-06T02:22:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.437+0000 7ffb2bfff700 1 -- 172.21.15.89:0/198751964 reap_dead start 2023-07-06T02:22:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.437+0000 7ffb31bfa700 1 -- 172.21.15.89:0/198751964 shutdown_connections 2023-07-06T02:22:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.437+0000 7ffb31bfa700 1 -- 172.21.15.89:0/198751964 >> 172.21.15.89:0/198751964 conn(0x7ffb2c0d0390 msgr2=0x7ffb2c0d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.437+0000 7ffb31bfa700 1 -- 172.21.15.89:0/198751964 shutdown_connections 2023-07-06T02:22:34.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.437+0000 7ffb31bfa700 1 -- 172.21.15.89:0/198751964 wait complete. 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.438+0000 7ffb31bfa700 1 Processor -- start 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.439+0000 7ffb31bfa700 1 -- start start 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.439+0000 7ffb31bfa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14003a30 0x7ffb2c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.439+0000 7ffb31bfa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c0b84b0 0x7ffb2c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb31bfa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb2c151d20 0x7ffb2c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb31bfa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c151890 con 0x7ffb2c151d20 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb31bfa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c150c80 con 0x7ffb14003a30 2023-07-06T02:22:34.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb31bfa700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c150f60 con 0x7ffb2c0b84b0 2023-07-06T02:22:34.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb2c151d20 0x7ffb2c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:34.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2affd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c0b84b0 0x7ffb2c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:34.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2b7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14003a30 0x7ffb2c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:34.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb2c151d20 0x7ffb2c0cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59572/0 (socket says 172.21.15.89:59572) 2023-07-06T02:22:34.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2affd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c0b84b0 0x7ffb2c14f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:54546/0 (socket says 172.21.15.89:54546) 2023-07-06T02:22:34.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 -- 172.21.15.89:0/2660866299 learned_addr learned my addr 172.21.15.89:0/2660866299 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:34.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 -- 172.21.15.89:0/2660866299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c0b84b0 msgr2=0x7ffb2c14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:34.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/2660866299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c0b84b0 0x7ffb2c14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 -- 172.21.15.89:0/2660866299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14003a30 msgr2=0x7ffb2c1505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:34.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/2660866299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14003a30 0x7ffb2c1505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 -- 172.21.15.89:0/2660866299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c151240 con 0x7ffb2c151d20 2023-07-06T02:22:34.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb2bfff700 1 --2- 172.21.15.89:0/2660866299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb2c151d20 0x7ffb2c0cbf20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffb140047d0 tx=0x7ffb24007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:34.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb13fff700 1 -- 172.21.15.89:0/2660866299 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24014410 con 0x7ffb2c151d20 2023-07-06T02:22:34.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.440+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c151520 con 0x7ffb2c151d20 2023-07-06T02:22:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.441+0000 7ffb13fff700 1 -- 172.21.15.89:0/2660866299 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24014570 con 0x7ffb2c151d20 2023-07-06T02:22:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.441+0000 7ffb13fff700 1 -- 172.21.15.89:0/2660866299 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24014920 con 0x7ffb2c151d20 2023-07-06T02:22:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.441+0000 7ffb13fff700 1 -- 172.21.15.89:0/2660866299 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffb24014a80 con 0x7ffb2c151d20 2023-07-06T02:22:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.441+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c14f7b0 con 0x7ffb2c151d20 2023-07-06T02:22:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.441+0000 7ffb13fff700 1 -- 172.21.15.89:0/2660866299 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb2401dc20 con 0x7ffb2c151d20 2023-07-06T02:22:34.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.442+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c151a00 con 0x7ffb2c151d20 2023-07-06T02:22:34.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.444+0000 7ffb13fff700 1 -- 172.21.15.89:0/2660866299 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ffb24009bd0 con 0x7ffb2c151d20 2023-07-06T02:22:34.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.520+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ffb2c074270 con 0x7ffb2c151d20 2023-07-06T02:22:34.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.527+0000 7ffb13fff700 1 -- 172.21.15.89:0/2660866299 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffb2401dde0 con 0x7ffb2c151d20 2023-07-06T02:22:34.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb2c151d20 msgr2=0x7ffb2c0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:34.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 --2- 172.21.15.89:0/2660866299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb2c151d20 0x7ffb2c0cbf20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffb140047d0 tx=0x7ffb24007610 comp rx=0 tx=0).stop 2023-07-06T02:22:34.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 shutdown_connections 2023-07-06T02:22:34.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 --2- 172.21.15.89:0/2660866299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb2c151d20 0x7ffb2c0cbf20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 --2- 172.21.15.89:0/2660866299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb14003a30 0x7ffb2c1505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 --2- 172.21.15.89:0/2660866299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb2c0b84b0 0x7ffb2c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:34.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 >> 172.21.15.89:0/2660866299 conn(0x7ffb2c0d0390 msgr2=0x7ffb2c0d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:34.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 shutdown_connections 2023-07-06T02:22:34.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:34.528+0000 7ffb31bfa700 1 -- 172.21.15.89:0/2660866299 wait complete. 2023-07-06T02:22:34.532 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2023-07-06T02:22:34.542 INFO:tasks.ceph:check_enable_crimson: False 2023-07-06T02:22:34.543 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-06T02:22:34.543 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:34.543 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-06T02:22:34.554 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:34.552+0000 7f60e02e5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T02:22:34.560 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:34.557+0000 7f5b0fb83200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T02:22:34.570 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:34.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-06T02:22:34.618 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:34.617+0000 7f60e02e5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T02:22:34.629 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:34.627+0000 7f5b0fb83200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T02:22:34.635 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:34.636 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-06T02:22:34.692 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:34.691+0000 7f60e02e5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T02:22:34.704 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:22:34.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-06T02:22:34.712 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:34.709+0000 7f5b0fb83200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T02:22:34.738 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:34.738 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-06T02:22:34.765 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:34.765 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-06T02:22:34.775 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:34.773+0000 7f60e02e5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T02:22:34.795 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:34.792+0000 7f5b0fb83200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T02:22:34.800 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:34.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-06T02:22:34.867 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:22:34.867 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-06T02:22:34.929 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:34.928+0000 7f60e02e5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T02:22:34.932 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 8b518e38-9c91-4acb-80d4-af070fb92b32 0 2023-07-06T02:22:34.937 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:34.934+0000 7f5b0fb83200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T02:22:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.093+0000 7f1afdbd8700 1 Processor -- start 2023-07-06T02:22:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.093+0000 7f1afdbd8700 1 -- start start 2023-07-06T02:22:35.097 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1afdbd8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af8156ab0 0x7f1af8160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.097 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1afdbd8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1af8160bb0 0x7f1af80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1afdbd8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af80cb3d0 con 0x7f1af81600b0 2023-07-06T02:22:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1afdbd8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af8156e80 con 0x7f1af8160bb0 2023-07-06T02:22:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1afdbd8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80ca370 con 0x7f1af8156ab0 2023-07-06T02:22:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1af77fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af8156ab0 0x7f1af8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1af6ffd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1af8160bb0 0x7f1af80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1af6ffd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1af8160bb0 0x7f1af80b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51210/0 (socket says 172.21.15.172:51210) 2023-07-06T02:22:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1af77fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af8156ab0 0x7f1af8160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:53116/0 (socket says 172.21.15.172:53116) 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.094+0000 7f1af6ffd700 1 -- 172.21.15.172:0/3632849730 learned_addr learned my addr 172.21.15.172:0/3632849730 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af6ffd700 1 -- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af8156ab0 msgr2=0x7f1af8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af6ffd700 1 --2- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af8156ab0 0x7f1af8160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af6ffd700 1 -- 172.21.15.172:0/3632849730 >> v1:172.21.15.172:6789/0 conn(0x7f1af81600b0 legacy=0x7f1af806d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af6ffd700 1 -- 172.21.15.172:0/3632849730 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80ca610 con 0x7f1af8160bb0 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af67fc700 1 -- 172.21.15.172:0/3632849730 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378837034 0 0) 0x7f1af80cb3d0 con 0x7f1af81600b0 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af77fe700 1 --2- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af8156ab0 0x7f1af8160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af6ffd700 1 --2- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8160bb0 0x7f1af80b1490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0098f0 tx=0x7f1aec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48fef3b9106b6ff3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af67fc700 1 -- 172.21.15.172:0/3632849730 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec011830 con 0x7f1af8160bb0 2023-07-06T02:22:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af67fc700 1 -- 172.21.15.172:0/3632849730 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec011990 con 0x7f1af8160bb0 2023-07-06T02:22:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1af67fc700 1 -- 172.21.15.172:0/3632849730 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec011d20 con 0x7f1af8160bb0 2023-07-06T02:22:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1afdbd8700 1 -- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8160bb0 msgr2=0x7f1af80b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1afdbd8700 1 --2- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8160bb0 0x7f1af80b1490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0098f0 tx=0x7f1aec009c00 comp rx=0 tx=0).stop 2023-07-06T02:22:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1afdbd8700 1 -- 172.21.15.172:0/3632849730 shutdown_connections 2023-07-06T02:22:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1afdbd8700 1 --2- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af8156ab0 0x7f1af8160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1afdbd8700 1 --2- 172.21.15.172:0/3632849730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8160bb0 0x7f1af80b1490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1afdbd8700 1 -- 172.21.15.172:0/3632849730 >> 172.21.15.172:0/3632849730 conn(0x7f1af80cc490 msgr2=0x7f1af80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.095+0000 7f1afdbd8700 1 -- 172.21.15.172:0/3632849730 shutdown_connections 2023-07-06T02:22:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.096+0000 7f1afdbd8700 1 -- 172.21.15.172:0/3632849730 wait complete. 2023-07-06T02:22:35.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.097+0000 7f1afdbd8700 1 Processor -- start 2023-07-06T02:22:35.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.097+0000 7f1afdbd8700 1 -- start start 2023-07-06T02:22:35.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8156ab0 0x7f1af81f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af81600b0 0x7f1af8155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af80c4f00 0x7f1af80c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af81f5df0 con 0x7f1af81600b0 2023-07-06T02:22:35.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80c6f80 con 0x7f1af8156ab0 2023-07-06T02:22:35.104 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af81f5830 con 0x7f1af80c4f00 2023-07-06T02:22:35.104 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8156ab0 0x7f1af81f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.104 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af7fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af80c4f00 0x7f1af80c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.104 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8156ab0 0x7f1af81f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51214/0 (socket says 172.21.15.172:51214) 2023-07-06T02:22:35.104 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 -- 172.21.15.172:0/2982231236 learned_addr learned my addr 172.21.15.172:0/2982231236 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:35.104 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 -- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af80c4f00 msgr2=0x7f1af80c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af80c4f00 0x7f1af80c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af6ffd700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af81600b0 0x7f1af8155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 -- 172.21.15.172:0/2982231236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af81600b0 msgr2=0x7f1af8155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af81600b0 0x7f1af8155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 -- 172.21.15.172:0/2982231236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80094a0 con 0x7f1af8156ab0 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af7fff700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af80c4f00 0x7f1af80c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1af77fe700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8156ab0 0x7f1af81f4f50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800ebe0 tx=0x7f1ae800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.105 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1adffff700 1 -- 172.21.15.172:0/2982231236 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8009800 con 0x7f1af8156ab0 2023-07-06T02:22:35.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1adffff700 1 -- 172.21.15.172:0/2982231236 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae800c910 con 0x7f1af8156ab0 2023-07-06T02:22:35.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec009550 con 0x7f1af8156ab0 2023-07-06T02:22:35.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1adffff700 1 -- 172.21.15.172:0/2982231236 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800cc80 con 0x7f1af8156ab0 2023-07-06T02:22:35.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.098+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af80c6c10 con 0x7f1af8156ab0 2023-07-06T02:22:35.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.099+0000 7f1adffff700 1 -- 172.21.15.172:0/2982231236 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ae8005d20 con 0x7f1af8156ab0 2023-07-06T02:22:35.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.099+0000 7f1adffff700 1 -- 172.21.15.172:0/2982231236 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1ae80137b0 con 0x7f1af8156ab0 2023-07-06T02:22:35.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.099+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af8074270 con 0x7f1af8156ab0 2023-07-06T02:22:35.107 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.101+0000 7f1adffff700 1 -- 172.21.15.172:0/2982231236 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f1ae8021060 con 0x7f1af8156ab0 2023-07-06T02:22:35.185 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.183+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8b518e38-9c91-4acb-80d4-af070fb92b32", "id": 0} v 0) v1 -- 0x7f1af80b5990 con 0x7f1af8156ab0 2023-07-06T02:22:35.190 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.189+0000 7f1adffff700 1 -- 172.21.15.172:0/2982231236 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8b518e38-9c91-4acb-80d4-af070fb92b32", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ae8017070 con 0x7f1af8156ab0 2023-07-06T02:22:35.191 INFO:teuthology.orchestra.run.smithi172.stdout:0 2023-07-06T02:22:35.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8156ab0 msgr2=0x7f1af81f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8156ab0 0x7f1af81f4f50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800ebe0 tx=0x7f1ae800eef0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 shutdown_connections 2023-07-06T02:22:35.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1af80c4f00 0x7f1af80c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af8156ab0 0x7f1af81f4f50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 --2- 172.21.15.172:0/2982231236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af81600b0 0x7f1af8155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 >> 172.21.15.172:0/2982231236 conn(0x7f1af80cc490 msgr2=0x7f1af80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 shutdown_connections 2023-07-06T02:22:35.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.190+0000 7f1afdbd8700 1 -- 172.21.15.172:0/2982231236 wait complete. 2023-07-06T02:22:35.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new f4e25425-59b1-484b-b38b-dd3b494a8756 1 2023-07-06T02:22:35.281 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:35.278+0000 7f5b0fb83200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T02:22:35.287 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:35.286+0000 7f60e02e5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T02:22:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.325+0000 7f7ebc2f7700 1 Processor -- start 2023-07-06T02:22:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.325+0000 7f7ebc2f7700 1 -- start start 2023-07-06T02:22:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7ebc2f7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7eb415c690 0x7f7eb40cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7ebc2f7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb40cbda0 0x7f7eb40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7ebc2f7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb40cab50 con 0x7f7eb40b19d0 2023-07-06T02:22:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7ebc2f7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb415de50 con 0x7f7eb415c690 2023-07-06T02:22:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7ebc2f7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb415db70 con 0x7f7eb40cbda0 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7eb9892700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb40cbda0 0x7f7eb40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7eba894700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7eb40b19d0 0x7f7eb406bfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56004/0 (socket says 172.21.15.172:56004) 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.326+0000 7f7eba894700 1 -- 172.21.15.172:0/1594062382 learned_addr learned my addr 172.21.15.172:0/1594062382 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eba093700 1 --2- 172.21.15.172:0/1594062382 >> v2:172.21.15.89:3300/0 conn(0x7f7eb415c690 0x7f7eb40cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9091700 1 -- 172.21.15.172:0/1594062382 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422413878 0 0) 0x7f7eb40cab50 con 0x7f7eb40b19d0 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9091700 1 -- 172.21.15.172:0/1594062382 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea00035b0 con 0x7f7eb40b19d0 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9892700 1 -- 172.21.15.172:0/1594062382 >> v2:172.21.15.89:3300/0 conn(0x7f7eb415c690 msgr2=0x7f7eb40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:35.330 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9892700 1 --2- 172.21.15.172:0/1594062382 >> v2:172.21.15.89:3300/0 conn(0x7f7eb415c690 0x7f7eb40cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.331 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9892700 1 -- 172.21.15.172:0/1594062382 >> v1:172.21.15.172:6789/0 conn(0x7f7eb40b19d0 legacy=0x7f7eb406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.331 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9892700 1 -- 172.21.15.172:0/1594062382 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb40b1e30 con 0x7f7eb40cbda0 2023-07-06T02:22:35.331 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9892700 1 --2- 172.21.15.172:0/1594062382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb40cbda0 0x7f7eb40b1490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009840 tx=0x7f7ea4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b0d6d91912e271 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.331 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9091700 1 -- 172.21.15.172:0/1594062382 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4011830 con 0x7f7eb40cbda0 2023-07-06T02:22:35.331 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9091700 1 -- 172.21.15.172:0/1594062382 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea4011990 con 0x7f7eb40cbda0 2023-07-06T02:22:35.331 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.327+0000 7f7eb9091700 1 -- 172.21.15.172:0/1594062382 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4011d20 con 0x7f7eb40cbda0 2023-07-06T02:22:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/1594062382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb40cbda0 msgr2=0x7f7eb40b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 --2- 172.21.15.172:0/1594062382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb40cbda0 0x7f7eb40b1490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009840 tx=0x7f7ea4009b50 comp rx=0 tx=0).stop 2023-07-06T02:22:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/1594062382 shutdown_connections 2023-07-06T02:22:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 --2- 172.21.15.172:0/1594062382 >> v2:172.21.15.89:3300/0 conn(0x7f7eb415c690 0x7f7eb40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 --2- 172.21.15.172:0/1594062382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb40cbda0 0x7f7eb40b1490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/1594062382 >> 172.21.15.172:0/1594062382 conn(0x7f7eb40cc490 msgr2=0x7f7eb40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/1594062382 shutdown_connections 2023-07-06T02:22:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.328+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/1594062382 wait complete. 2023-07-06T02:22:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.329+0000 7f7ebc2f7700 1 Processor -- start 2023-07-06T02:22:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.329+0000 7f7ebc2f7700 1 -- start start 2023-07-06T02:22:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7ebc2f7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb40b19d0 0x7f7eb41eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7ebc2f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eb40cbda0 0x7f7eb40c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7ebc2f7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb415c690 0x7f7eb4157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7ebc2f7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40c6100 con 0x7f7eb40cbda0 2023-07-06T02:22:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7ebc2f7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb41ef290 con 0x7f7eb40b19d0 2023-07-06T02:22:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7ebc2f7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb41ef570 con 0x7f7eb415c690 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb415c690 0x7f7eb4157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb415c690 0x7f7eb4157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:53138/0 (socket says 172.21.15.172:53138) 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba093700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb40b19d0 0x7f7eb41eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 -- 172.21.15.172:0/893379616 learned_addr learned my addr 172.21.15.172:0/893379616 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eb9892700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eb40cbda0 0x7f7eb40c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 -- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb40b19d0 msgr2=0x7f7eb41eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb40b19d0 0x7f7eb41eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 -- 172.21.15.172:0/893379616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eb40cbda0 msgr2=0x7f7eb40c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eb40cbda0 0x7f7eb40c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 -- 172.21.15.172:0/893379616 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40094a0 con 0x7f7eb415c690 2023-07-06T02:22:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba093700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb40b19d0 0x7f7eb41eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.330+0000 7f7eba894700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb415c690 0x7f7eb4157e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00edd0 tx=0x7f7eac00c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.331+0000 7f7eaaffd700 1 -- 172.21.15.172:0/893379616 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac011470 con 0x7f7eb415c690 2023-07-06T02:22:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.331+0000 7f7eaaffd700 1 -- 172.21.15.172:0/893379616 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac0115d0 con 0x7f7eb415c690 2023-07-06T02:22:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.331+0000 7f7eaaffd700 1 -- 172.21.15.172:0/893379616 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac011960 con 0x7f7eb415c690 2023-07-06T02:22:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.331+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb41ef850 con 0x7f7eb415c690 2023-07-06T02:22:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.331+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb41efb30 con 0x7f7eb415c690 2023-07-06T02:22:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.331+0000 7f7eaaffd700 1 -- 172.21.15.172:0/893379616 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7eac011ac0 con 0x7f7eb415c690 2023-07-06T02:22:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.331+0000 7f7eaaffd700 1 -- 172.21.15.172:0/893379616 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f7eac0103d0 con 0x7f7eb415c690 2023-07-06T02:22:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.332+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb41f0070 con 0x7f7eb415c690 2023-07-06T02:22:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.334+0000 7f7eaaffd700 1 -- 172.21.15.172:0/893379616 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7eac039b00 con 0x7f7eb415c690 2023-07-06T02:22:35.356 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:35.353+0000 7f5b0fb83200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T02:22:35.366 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:35.364+0000 7f60e02e5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T02:22:35.419 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.418+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f4e25425-59b1-484b-b38b-dd3b494a8756", "id": 1} v 0) v1 -- 0x7f7eb4074270 con 0x7f7eb415c690 2023-07-06T02:22:35.427 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.426+0000 7f7eaaffd700 1 -- 172.21.15.172:0/893379616 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4e25425-59b1-484b-b38b-dd3b494a8756", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7eac028030 con 0x7f7eb415c690 2023-07-06T02:22:35.428 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-07-06T02:22:35.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb415c690 msgr2=0x7f7eb4157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb415c690 0x7f7eb4157e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00edd0 tx=0x7f7eac00c550 comp rx=0 tx=0).stop 2023-07-06T02:22:35.430 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 shutdown_connections 2023-07-06T02:22:35.430 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7eb415c690 0x7f7eb4157e00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.430 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb40b19d0 0x7f7eb41eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.430 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 --2- 172.21.15.172:0/893379616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eb40cbda0 0x7f7eb40c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.430 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 >> 172.21.15.172:0/893379616 conn(0x7f7eb40cc490 msgr2=0x7f7eb40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.431 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 shutdown_connections 2023-07-06T02:22:35.431 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.427+0000 7f7ebc2f7700 1 -- 172.21.15.172:0/893379616 wait complete. 2023-07-06T02:22:35.441 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new c0faa4f3-24e2-4a22-8c3f-e4c021271ef6 2 2023-07-06T02:22:35.485 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:35.482+0000 7f5b0fb83200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T02:22:35.496 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:35.494+0000 7f60e02e5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T02:22:35.562 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.560+0000 7fc1e7e41700 1 Processor -- start 2023-07-06T02:22:35.562 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.560+0000 7fc1e7e41700 1 -- start start 2023-07-06T02:22:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.561+0000 7fc1e7e41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1e00caf80 0x7fc1e006bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.561+0000 7fc1e7e41700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e006a500 con 0x7fc1e0071630 2023-07-06T02:22:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.561+0000 7fc1e7e41700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e006ec60 con 0x7fc1e015a3e0 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.561+0000 7fc1e7e41700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e006f190 con 0x7fc1e00caf80 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.561+0000 7fc1e5bdd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc1e015a3e0 0x7fc1e0070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56012/0 (socket says 172.21.15.172:56012) 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.561+0000 7fc1e5bdd700 1 -- 172.21.15.172:0/3706447401 learned_addr learned my addr 172.21.15.172:0/3706447401 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.561+0000 7fc1e63de700 1 --2- 172.21.15.172:0/3706447401 >> v2:172.21.15.89:3300/0 conn(0x7fc1e00caf80 0x7fc1e006bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e4bdb700 1 -- 172.21.15.172:0/3706447401 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465256579 0 0) 0x7fc1e006ec60 con 0x7fc1e015a3e0 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e4bdb700 1 -- 172.21.15.172:0/3706447401 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1cc0035b0 con 0x7fc1e015a3e0 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e4bdb700 1 -- 172.21.15.172:0/3706447401 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875444874 0 0) 0x7fc1e006a500 con 0x7fc1e0071630 2023-07-06T02:22:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e4bdb700 1 -- 172.21.15.172:0/3706447401 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e006ec60 con 0x7fc1e0071630 2023-07-06T02:22:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e63de700 1 -- 172.21.15.172:0/3706447401 >> v1:172.21.15.172:6789/0 conn(0x7fc1e015a3e0 legacy=0x7fc1e0070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e63de700 1 -- 172.21.15.172:0/3706447401 >> v1:172.21.15.89:6789/0 conn(0x7fc1e0071630 legacy=0x7fc1e015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e63de700 1 -- 172.21.15.172:0/3706447401 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00cb980 con 0x7fc1e00caf80 2023-07-06T02:22:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e63de700 1 --2- 172.21.15.172:0/3706447401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e00caf80 0x7fc1e006bcd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009d90 tx=0x7fc1d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecfb51e91ba4dc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e4bdb700 1 -- 172.21.15.172:0/3706447401 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800e070 con 0x7fc1e00caf80 2023-07-06T02:22:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e4bdb700 1 -- 172.21.15.172:0/3706447401 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d800bb20 con 0x7fc1e00caf80 2023-07-06T02:22:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.562+0000 7fc1e4bdb700 1 -- 172.21.15.172:0/3706447401 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8012430 con 0x7fc1e00caf80 2023-07-06T02:22:35.569 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.563+0000 7fc1e7e41700 1 -- 172.21.15.172:0/3706447401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e00caf80 msgr2=0x7fc1e006bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.569 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.563+0000 7fc1e7e41700 1 --2- 172.21.15.172:0/3706447401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e00caf80 0x7fc1e006bcd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009d90 tx=0x7fc1d800b330 comp rx=0 tx=0).stop 2023-07-06T02:22:35.569 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.563+0000 7fc1e7e41700 1 -- 172.21.15.172:0/3706447401 shutdown_connections 2023-07-06T02:22:35.569 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.563+0000 7fc1e7e41700 1 --2- 172.21.15.172:0/3706447401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e00caf80 0x7fc1e006bcd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.570 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.563+0000 7fc1e7e41700 1 -- 172.21.15.172:0/3706447401 >> 172.21.15.172:0/3706447401 conn(0x7fc1e00cf1c0 msgr2=0x7fc1e00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.570 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.563+0000 7fc1e7e41700 1 -- 172.21.15.172:0/3706447401 shutdown_connections 2023-07-06T02:22:35.570 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.563+0000 7fc1e7e41700 1 -- 172.21.15.172:0/3706447401 wait complete. 2023-07-06T02:22:35.570 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.564+0000 7fc1e7e41700 1 Processor -- start 2023-07-06T02:22:35.570 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.564+0000 7fc1e7e41700 1 -- start start 2023-07-06T02:22:35.570 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e7e41700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1e0071630 0x7fc1e01e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.570 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e7e41700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e00caf80 0x7fc1e00c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e7e41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e015a3e0 0x7fc1e00b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e7e41700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00aba20 con 0x7fc1e015a3e0 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e7e41700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00abd00 con 0x7fc1e00caf80 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e7e41700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00abfe0 con 0x7fc1e0071630 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e00caf80 0x7fc1e00c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e5bdd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1e0071630 0x7fc1e01e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e00caf80 0x7fc1e00c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51240/0 (socket says 172.21.15.172:51240) 2023-07-06T02:22:35.571 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 -- 172.21.15.172:0/1211792071 learned_addr learned my addr 172.21.15.172:0/1211792071 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:35.572 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e63de700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e015a3e0 0x7fc1e00b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.572 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 -- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1e0071630 msgr2=0x7fc1e01e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.572 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1e0071630 0x7fc1e01e6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.572 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 -- 172.21.15.172:0/1211792071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e015a3e0 msgr2=0x7fc1e00b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.572 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e015a3e0 0x7fc1e00b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.572 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 -- 172.21.15.172:0/1211792071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009a70 con 0x7fc1e00caf80 2023-07-06T02:22:35.572 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e5bdd700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1e0071630 0x7fc1e01e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1e53dc700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e00caf80 0x7fc1e00c52f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c4d0 tx=0x7fc1d000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.565+0000 7fc1d67fc700 1 -- 172.21.15.172:0/1211792071 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00052d0 con 0x7fc1e00caf80 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.566+0000 7fc1d67fc700 1 -- 172.21.15.172:0/1211792071 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0005430 con 0x7fc1e00caf80 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.566+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e00ac2c0 con 0x7fc1e00caf80 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.566+0000 7fc1d67fc700 1 -- 172.21.15.172:0/1211792071 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0003880 con 0x7fc1e00caf80 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.566+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e0158e70 con 0x7fc1e00caf80 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.566+0000 7fc1d67fc700 1 -- 172.21.15.172:0/1211792071 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc1d0003ce0 con 0x7fc1e00caf80 2023-07-06T02:22:35.573 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.566+0000 7fc1d67fc700 1 -- 172.21.15.172:0/1211792071 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fc1d0011d10 con 0x7fc1e00caf80 2023-07-06T02:22:35.574 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.566+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e0074270 con 0x7fc1e00caf80 2023-07-06T02:22:35.574 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:35.565+0000 7f5b0fb83200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T02:22:35.574 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.569+0000 7fc1d67fc700 1 -- 172.21.15.172:0/1211792071 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc1d0021060 con 0x7fc1e00caf80 2023-07-06T02:22:35.581 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:35.579+0000 7f60e02e5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T02:22:35.653 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.651+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c0faa4f3-24e2-4a22-8c3f-e4c021271ef6", "id": 2} v 0) v1 -- 0x7fc1e006b810 con 0x7fc1e00caf80 2023-07-06T02:22:35.657 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.655+0000 7fc1d67fc700 1 -- 172.21.15.172:0/1211792071 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c0faa4f3-24e2-4a22-8c3f-e4c021271ef6", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc1d0017070 con 0x7fc1e00caf80 2023-07-06T02:22:35.657 INFO:teuthology.orchestra.run.smithi172.stdout:2 2023-07-06T02:22:35.658 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e00caf80 msgr2=0x7fc1e00c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.658 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e00caf80 0x7fc1e00c52f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c4d0 tx=0x7fc1d000c7e0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.658 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 shutdown_connections 2023-07-06T02:22:35.658 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1e015a3e0 0x7fc1e00b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.659 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1e0071630 0x7fc1e01e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.659 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 --2- 172.21.15.172:0/1211792071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e00caf80 0x7fc1e00c52f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.659 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 >> 172.21.15.172:0/1211792071 conn(0x7fc1e00cf1c0 msgr2=0x7fc1e00b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.659 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 shutdown_connections 2023-07-06T02:22:35.659 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.656+0000 7fc1e7e41700 1 -- 172.21.15.172:0/1211792071 wait complete. 2023-07-06T02:22:35.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new b1e33d2f-29d4-4d0e-b89e-01382e19c83f 3 2023-07-06T02:22:35.793 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.791+0000 7f6afb2e1700 1 Processor -- start 2023-07-06T02:22:35.793 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.791+0000 7f6afb2e1700 1 -- start start 2023-07-06T02:22:35.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6afb2e1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6af41600b0 0x7f6af40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6afb2e1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6afb2e1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af40cb3d0 con 0x7f6af4156ab0 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6afb2e1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af40b1e10 con 0x7f6af41600b0 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6afb2e1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4156e80 con 0x7f6af40b19d0 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af987e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af907d700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6af4156ab0 0x7f6af4160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56024/0 (socket says 172.21.15.172:56024) 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af907d700 1 -- 172.21.15.172:0/3979392956 learned_addr learned my addr 172.21.15.172:0/3979392956 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-07-06T02:22:35.795 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af887c700 1 --2- 172.21.15.172:0/3979392956 >> v2:172.21.15.89:3300/0 conn(0x7f6af41600b0 0x7f6af40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.796 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af987e700 1 -- 172.21.15.172:0/3979392956 >> v2:172.21.15.89:3300/0 conn(0x7f6af41600b0 msgr2=0x7f6af40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:35.796 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af987e700 1 --2- 172.21.15.172:0/3979392956 >> v2:172.21.15.89:3300/0 conn(0x7f6af41600b0 0x7f6af40b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.796 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af987e700 1 -- 172.21.15.172:0/3979392956 >> v1:172.21.15.172:6789/0 conn(0x7f6af4156ab0 legacy=0x7f6af4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.796 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6af987e700 1 -- 172.21.15.172:0/3979392956 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40ca370 con 0x7f6af40b19d0 2023-07-06T02:22:35.796 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.792+0000 7f6aebfff700 1 -- 172.21.15.172:0/3979392956 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515444702 0 0) 0x7f6af40cb3d0 con 0x7f6af4156ab0 2023-07-06T02:22:35.796 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6af987e700 1 --2- 172.21.15.172:0/3979392956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40cd700 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009d90 tx=0x7f6aec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c720d23f5b165ff4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.796 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6aebfff700 1 -- 172.21.15.172:0/3979392956 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00e070 con 0x7f6af40b19d0 2023-07-06T02:22:35.797 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6aebfff700 1 -- 172.21.15.172:0/3979392956 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec00bb20 con 0x7f6af40b19d0 2023-07-06T02:22:35.797 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6aebfff700 1 -- 172.21.15.172:0/3979392956 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec012430 con 0x7f6af40b19d0 2023-07-06T02:22:35.799 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3979392956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 msgr2=0x7f6af40cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.799 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 --2- 172.21.15.172:0/3979392956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40cd700 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009d90 tx=0x7f6aec00b330 comp rx=0 tx=0).stop 2023-07-06T02:22:35.799 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3979392956 shutdown_connections 2023-07-06T02:22:35.799 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 --2- 172.21.15.172:0/3979392956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40cd700 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.799 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 --2- 172.21.15.172:0/3979392956 >> v2:172.21.15.89:3300/0 conn(0x7f6af41600b0 0x7f6af40b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.800 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3979392956 >> 172.21.15.172:0/3979392956 conn(0x7f6af40cc490 msgr2=0x7f6af40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.800 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3979392956 shutdown_connections 2023-07-06T02:22:35.800 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.793+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3979392956 wait complete. 2023-07-06T02:22:35.800 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 Processor -- start 2023-07-06T02:22:35.800 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 -- start start 2023-07-06T02:22:35.800 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.800 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af4156ab0 0x7f6af41e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41600b0 0x7f6af41e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af406b480 con 0x7f6af4156ab0 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40b3780 con 0x7f6af41600b0 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6afb2e1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40b57d0 con 0x7f6af40b19d0 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.795+0000 7f6af987e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41600b0 0x7f6af41e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41600b0 0x7f6af41e6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51254/0 (socket says 172.21.15.172:51254) 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 -- 172.21.15.172:0/3418282485 learned_addr learned my addr 172.21.15.172:0/3418282485 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:35.801 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af907d700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.802 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af887c700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af4156ab0 0x7f6af41e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:35.802 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 -- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 msgr2=0x7f6af40b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.802 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.802 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 -- 172.21.15.172:0/3418282485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af4156ab0 msgr2=0x7f6af41e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.802 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af4156ab0 0x7f6af41e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.802 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 -- 172.21.15.172:0/3418282485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec009a70 con 0x7f6af41600b0 2023-07-06T02:22:35.802 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6af987e700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41600b0 0x7f6af41e6310 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6af4156fc0 tx=0x7f6aec004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6ae9ffb700 1 -- 172.21.15.172:0/3418282485 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec012430 con 0x7f6af41600b0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6ae9ffb700 1 -- 172.21.15.172:0/3418282485 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec020070 con 0x7f6af41600b0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af40b54f0 con 0x7f6af41600b0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6ae9ffb700 1 -- 172.21.15.172:0/3418282485 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec004280 con 0x7f6af41600b0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.796+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af406b0a0 con 0x7f6af41600b0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.797+0000 7f6ae9ffb700 1 -- 172.21.15.172:0/3418282485 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6aec0046e0 con 0x7f6af41600b0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.797+0000 7f6ae9ffb700 1 -- 172.21.15.172:0/3418282485 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f6aec022070 con 0x7f6af41600b0 2023-07-06T02:22:35.803 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.797+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af406b5c0 con 0x7f6af41600b0 2023-07-06T02:22:35.804 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.799+0000 7f6ae9ffb700 1 -- 172.21.15.172:0/3418282485 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6af406b5c0 con 0x7f6af41600b0 2023-07-06T02:22:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.881+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b1e33d2f-29d4-4d0e-b89e-01382e19c83f", "id": 3} v 0) v1 -- 0x7f6af4066fb0 con 0x7f6af41600b0 2023-07-06T02:22:35.887 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.885+0000 7f6ae9ffb700 1 -- 172.21.15.172:0/3418282485 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1e33d2f-29d4-4d0e-b89e-01382e19c83f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6aec02c060 con 0x7f6af41600b0 2023-07-06T02:22:35.887 INFO:teuthology.orchestra.run.smithi172.stdout:3 2023-07-06T02:22:35.888 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41600b0 msgr2=0x7f6af41e6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:35.888 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41600b0 0x7f6af41e6310 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6af4156fc0 tx=0x7f6aec004ad0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.888 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 shutdown_connections 2023-07-06T02:22:35.888 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41600b0 0x7f6af41e6310 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.889 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6af40b19d0 0x7f6af40b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.889 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 --2- 172.21.15.172:0/3418282485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af4156ab0 0x7f6af41e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:35.889 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 >> 172.21.15.172:0/3418282485 conn(0x7f6af40cc490 msgr2=0x7f6af40b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:35.889 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 shutdown_connections 2023-07-06T02:22:35.889 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:35.886+0000 7f6afb2e1700 1 -- 172.21.15.172:0/3418282485 wait complete. 2023-07-06T02:22:35.899 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 5a9e51c3-f007-45af-90c6-0925a630e939 4 2023-07-06T02:22:36.000 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:35.998+0000 7f5b0fb83200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T02:22:36.019 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.017+0000 7f5c7da29700 1 Processor -- start 2023-07-06T02:22:36.019 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.017+0000 7f5c7da29700 1 -- start start 2023-07-06T02:22:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c7da29700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c780b5e50 0x7f5c78070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c7da29700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c78070870 0x7f5c780cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c7da29700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780cb150 con 0x7f5c7815c610 2023-07-06T02:22:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c7da29700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780d05e0 con 0x7f5c780b5e50 2023-07-06T02:22:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c7da29700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780cb430 con 0x7f5c78070870 2023-07-06T02:22:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c76ffd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5c7815c610 0x7f5c780cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56036/0 (socket says 172.21.15.172:56036) 2023-07-06T02:22:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c76ffd700 1 -- 172.21.15.172:0/164919236 learned_addr learned my addr 172.21.15.172:0/164919236 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-07-06T02:22:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c78070870 0x7f5c780cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c767fc700 1 --2- 172.21.15.172:0/164919236 >> v2:172.21.15.89:3300/0 conn(0x7f5c780b5e50 0x7f5c78070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c75ffb700 1 -- 172.21.15.172:0/164919236 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025538454 0 0) 0x7f5c780cb150 con 0x7f5c7815c610 2023-07-06T02:22:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c75ffb700 1 -- 172.21.15.172:0/164919236 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c600035b0 con 0x7f5c7815c610 2023-07-06T02:22:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c777fe700 1 -- 172.21.15.172:0/164919236 >> v2:172.21.15.89:3300/0 conn(0x7f5c780b5e50 msgr2=0x7f5c78070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c777fe700 1 --2- 172.21.15.172:0/164919236 >> v2:172.21.15.89:3300/0 conn(0x7f5c780b5e50 0x7f5c78070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c777fe700 1 -- 172.21.15.172:0/164919236 >> v1:172.21.15.172:6789/0 conn(0x7f5c7815c610 legacy=0x7f5c780cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.018+0000 7f5c777fe700 1 -- 172.21.15.172:0/164919236 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78070cb0 con 0x7f5c78070870 2023-07-06T02:22:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c777fe700 1 --2- 172.21.15.172:0/164919236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c78070870 0x7f5c780cb8c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ff244d7154343c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c75ffb700 1 -- 172.21.15.172:0/164919236 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000e070 con 0x7f5c78070870 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c75ffb700 1 -- 172.21.15.172:0/164919236 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000bae0 con 0x7f5c78070870 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c75ffb700 1 -- 172.21.15.172:0/164919236 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000be90 con 0x7f5c78070870 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 -- 172.21.15.172:0/164919236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c78070870 msgr2=0x7f5c780cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 --2- 172.21.15.172:0/164919236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c78070870 0x7f5c780cb8c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7000b330 comp rx=0 tx=0).stop 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 -- 172.21.15.172:0/164919236 shutdown_connections 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 --2- 172.21.15.172:0/164919236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c78070870 0x7f5c780cb8c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 --2- 172.21.15.172:0/164919236 >> v2:172.21.15.89:3300/0 conn(0x7f5c780b5e50 0x7f5c78070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 -- 172.21.15.172:0/164919236 >> 172.21.15.172:0/164919236 conn(0x7f5c780d0b00 msgr2=0x7f5c780d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 -- 172.21.15.172:0/164919236 shutdown_connections 2023-07-06T02:22:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.019+0000 7f5c7da29700 1 -- 172.21.15.172:0/164919236 wait complete. 2023-07-06T02:22:36.027 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.021+0000 7f5c7da29700 1 Processor -- start 2023-07-06T02:22:36.027 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.021+0000 7f5c7da29700 1 -- start start 2023-07-06T02:22:36.027 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c7da29700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c78070870 0x7f5c7815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.027 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c7da29700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c780b5e50 0x7f5c781560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.027 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c7da29700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c7815c610 0x7f5c780c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.027 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c7da29700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780cb0d0 con 0x7f5c78070870 2023-07-06T02:22:36.027 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c7da29700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c4a20 con 0x7f5c780b5e50 2023-07-06T02:22:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c7da29700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815cf20 con 0x7f5c7815c610 2023-07-06T02:22:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c767fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c780b5e50 0x7f5c781560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c767fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c780b5e50 0x7f5c781560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51264/0 (socket says 172.21.15.172:51264) 2023-07-06T02:22:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c7815c610 0x7f5c780c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c767fc700 1 -- 172.21.15.172:0/3550844506 learned_addr learned my addr 172.21.15.172:0/3550844506 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.022+0000 7f5c76ffd700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c78070870 0x7f5c7815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c777fe700 1 -- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c780b5e50 msgr2=0x7f5c781560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c777fe700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c780b5e50 0x7f5c781560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c777fe700 1 -- 172.21.15.172:0/3550844506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c78070870 msgr2=0x7f5c7815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c777fe700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c78070870 0x7f5c7815c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c777fe700 1 -- 172.21.15.172:0/3550844506 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70009a70 con 0x7f5c7815c610 2023-07-06T02:22:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c767fc700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c780b5e50 0x7f5c781560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c76ffd700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c78070870 0x7f5c7815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c777fe700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c7815c610 0x7f5c780c4d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c70015000 tx=0x7f5c70004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c5ffff700 1 -- 172.21.15.172:0/3550844506 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700044c0 con 0x7f5c7815c610 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c5ffff700 1 -- 172.21.15.172:0/3550844506 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70004660 con 0x7f5c7815c610 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780c4740 con 0x7f5c7815c610 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c5ffff700 1 -- 172.21.15.172:0/3550844506 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70012430 con 0x7f5c7815c610 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780c8ff0 con 0x7f5c7815c610 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c5ffff700 1 -- 172.21.15.172:0/3550844506 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c70012890 con 0x7f5c7815c610 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.023+0000 7f5c5ffff700 1 -- 172.21.15.172:0/3550844506 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f5c7001dde0 con 0x7f5c7815c610 2023-07-06T02:22:36.030 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.024+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78074270 con 0x7f5c7815c610 2023-07-06T02:22:36.031 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.026+0000 7f5c5ffff700 1 -- 172.21.15.172:0/3550844506 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5c7001d420 con 0x7f5c7815c610 2023-07-06T02:22:36.060 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:36.058+0000 7f60e02e5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T02:22:36.111 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.110+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a9e51c3-f007-45af-90c6-0925a630e939", "id": 4} v 0) v1 -- 0x7f5c780b34c0 con 0x7f5c7815c610 2023-07-06T02:22:36.115 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.113+0000 7f5c5ffff700 1 -- 172.21.15.172:0/3550844506 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a9e51c3-f007-45af-90c6-0925a630e939", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c70012b40 con 0x7f5c7815c610 2023-07-06T02:22:36.115 INFO:teuthology.orchestra.run.smithi172.stdout:4 2023-07-06T02:22:36.116 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.114+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c7815c610 msgr2=0x7f5c780c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.117 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.114+0000 7f5c7da29700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c7815c610 0x7f5c780c4d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c70015000 tx=0x7f5c70004ad0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.117 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.115+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 shutdown_connections 2023-07-06T02:22:36.117 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.115+0000 7f5c7da29700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c7815c610 0x7f5c780c4d90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.117 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.115+0000 7f5c7da29700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c78070870 0x7f5c7815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.117 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.115+0000 7f5c7da29700 1 --2- 172.21.15.172:0/3550844506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c780b5e50 0x7f5c781560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.117 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.115+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 >> 172.21.15.172:0/3550844506 conn(0x7f5c780d0b00 msgr2=0x7f5c780ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.117 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.115+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 shutdown_connections 2023-07-06T02:22:36.118 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.115+0000 7f5c7da29700 1 -- 172.21.15.172:0/3550844506 wait complete. 2023-07-06T02:22:36.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new e689f34a-8b34-4de0-a752-ae331230a4d4 5 2023-07-06T02:22:36.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.249+0000 7f3298f20700 1 Processor -- start 2023-07-06T02:22:36.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.249+0000 7f3298f20700 1 -- start start 2023-07-06T02:22:36.253 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3298f20700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f3294160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.253 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3298f20700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3294160bb0 0x7f32940b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.253 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3298f20700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32940cb3d0 con 0x7f32941600b0 2023-07-06T02:22:36.253 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3298f20700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3294156e80 con 0x7f3294160bb0 2023-07-06T02:22:36.253 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3298f20700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32940ca370 con 0x7f3294156ab0 2023-07-06T02:22:36.253 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3292d9d700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f32941600b0 0x7f329406d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56048/0 (socket says 172.21.15.172:56048) 2023-07-06T02:22:36.254 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f329259c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f3294160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.254 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3292d9d700 1 -- 172.21.15.172:0/1503133278 learned_addr learned my addr 172.21.15.172:0/1503133278 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-07-06T02:22:36.254 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f3291d9b700 1 --2- 172.21.15.172:0/1503133278 >> v2:172.21.15.89:3300/0 conn(0x7f3294160bb0 0x7f32940b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.254 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f329159a700 1 -- 172.21.15.172:0/1503133278 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911626270 0 0) 0x7f32940cb3d0 con 0x7f32941600b0 2023-07-06T02:22:36.254 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f329159a700 1 -- 172.21.15.172:0/1503133278 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f327c0035b0 con 0x7f32941600b0 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f329259c700 1 -- 172.21.15.172:0/1503133278 >> v2:172.21.15.89:3300/0 conn(0x7f3294160bb0 msgr2=0x7f32940b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.250+0000 7f329259c700 1 --2- 172.21.15.172:0/1503133278 >> v2:172.21.15.89:3300/0 conn(0x7f3294160bb0 0x7f32940b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f329259c700 1 -- 172.21.15.172:0/1503133278 >> v1:172.21.15.172:6789/0 conn(0x7f32941600b0 legacy=0x7f329406d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f329259c700 1 -- 172.21.15.172:0/1503133278 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940ca610 con 0x7f3294156ab0 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f329259c700 1 --2- 172.21.15.172:0/1503133278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f3294160670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3284009840 tx=0x7f3284009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc78cdfd0c5774c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f329159a700 1 -- 172.21.15.172:0/1503133278 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284011830 con 0x7f3294156ab0 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f329159a700 1 -- 172.21.15.172:0/1503133278 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284011990 con 0x7f3294156ab0 2023-07-06T02:22:36.255 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f329159a700 1 -- 172.21.15.172:0/1503133278 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284011d20 con 0x7f3294156ab0 2023-07-06T02:22:36.258 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f3298f20700 1 -- 172.21.15.172:0/1503133278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 msgr2=0x7f3294160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.258 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.251+0000 7f3298f20700 1 --2- 172.21.15.172:0/1503133278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f3294160670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3284009840 tx=0x7f3284009b50 comp rx=0 tx=0).stop 2023-07-06T02:22:36.258 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.252+0000 7f3298f20700 1 -- 172.21.15.172:0/1503133278 shutdown_connections 2023-07-06T02:22:36.258 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.252+0000 7f3298f20700 1 --2- 172.21.15.172:0/1503133278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f3294160670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.258 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.252+0000 7f3298f20700 1 --2- 172.21.15.172:0/1503133278 >> v2:172.21.15.89:3300/0 conn(0x7f3294160bb0 0x7f32940b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.258 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.252+0000 7f3298f20700 1 -- 172.21.15.172:0/1503133278 >> 172.21.15.172:0/1503133278 conn(0x7f32940cc490 msgr2=0x7f32940cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.252+0000 7f3298f20700 1 -- 172.21.15.172:0/1503133278 shutdown_connections 2023-07-06T02:22:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.252+0000 7f3298f20700 1 -- 172.21.15.172:0/1503133278 wait complete. 2023-07-06T02:22:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.253+0000 7f3298f20700 1 Processor -- start 2023-07-06T02:22:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.253+0000 7f3298f20700 1 -- start start 2023-07-06T02:22:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f3298f20700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f32941f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f3298f20700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32941600b0 0x7f32940abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f3298f20700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32940ac2d0 0x7f32940c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f3298f20700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32940ac770 con 0x7f32940ac2d0 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f3298f20700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32940b1050 con 0x7f32941600b0 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f3298f20700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32940affb0 con 0x7f3294156ab0 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f329259c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f32941f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f3291d9b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32941600b0 0x7f32940abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f329259c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f32941f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:53182/0 (socket says 172.21.15.172:53182) 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.254+0000 7f329259c700 1 -- 172.21.15.172:0/1054815234 learned_addr learned my addr 172.21.15.172:0/1054815234 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:36.260 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3292d9d700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32940ac2d0 0x7f32940c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f329259c700 1 -- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32941600b0 msgr2=0x7f32940abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f329259c700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32941600b0 0x7f32940abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f329259c700 1 -- 172.21.15.172:0/1054815234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32940ac2d0 msgr2=0x7f32940c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f329259c700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32940ac2d0 0x7f32940c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f329259c700 1 -- 172.21.15.172:0/1054815234 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840094a0 con 0x7f3294156ab0 2023-07-06T02:22:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3291d9b700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32941600b0 0x7f32940abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f329259c700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f32941f14b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3284000c00 tx=0x7f3284004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3282ffd700 1 -- 172.21.15.172:0/1054815234 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284004590 con 0x7f3294156ab0 2023-07-06T02:22:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3282ffd700 1 -- 172.21.15.172:0/1054815234 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328401f070 con 0x7f3294156ab0 2023-07-06T02:22:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3282ffd700 1 -- 172.21.15.172:0/1054815234 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32840038e0 con 0x7f3294156ab0 2023-07-06T02:22:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32940b0290 con 0x7f3294156ab0 2023-07-06T02:22:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32940b0850 con 0x7f3294156ab0 2023-07-06T02:22:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3282ffd700 1 -- 172.21.15.172:0/1054815234 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3284003d40 con 0x7f3294156ab0 2023-07-06T02:22:36.263 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.255+0000 7f3282ffd700 1 -- 172.21.15.172:0/1054815234 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f3284021070 con 0x7f3294156ab0 2023-07-06T02:22:36.263 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.256+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3294074270 con 0x7f3294156ab0 2023-07-06T02:22:36.263 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.260+0000 7f3282ffd700 1 -- 172.21.15.172:0/1054815234 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3284045b00 con 0x7f3294156ab0 2023-07-06T02:22:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.343+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e689f34a-8b34-4de0-a752-ae331230a4d4", "id": 5} v 0) v1 -- 0x7f32940b5990 con 0x7f3294156ab0 2023-07-06T02:22:36.351 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.350+0000 7f3282ffd700 1 -- 172.21.15.172:0/1054815234 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e689f34a-8b34-4de0-a752-ae331230a4d4", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3284034030 con 0x7f3294156ab0 2023-07-06T02:22:36.352 INFO:teuthology.orchestra.run.smithi172.stdout:5 2023-07-06T02:22:36.353 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 msgr2=0x7f32941f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.353 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f32941f14b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3284000c00 tx=0x7f3284004750 comp rx=0 tx=0).stop 2023-07-06T02:22:36.353 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 shutdown_connections 2023-07-06T02:22:36.353 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32940ac2d0 0x7f32940c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.353 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3294156ab0 0x7f32941f14b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.353 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 --2- 172.21.15.172:0/1054815234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32941600b0 0x7f32940abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.354 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 >> 172.21.15.172:0/1054815234 conn(0x7f32940cc490 msgr2=0x7f329415db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.354 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 shutdown_connections 2023-07-06T02:22:36.354 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.351+0000 7f3298f20700 1 -- 172.21.15.172:0/1054815234 wait complete. 2023-07-06T02:22:36.365 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 664b96bb-7c04-4a18-b0ce-d96f5b0fffb5 6 2023-07-06T02:22:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.522+0000 7f89953d3700 1 Processor -- start 2023-07-06T02:22:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.522+0000 7f89953d3700 1 -- start start 2023-07-06T02:22:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f89953d3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8990070f20 0x7f89900713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f89953d3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8990160000 0x7f8990160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f89953d3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89900cb9f0 con 0x7f899015c500 2023-07-06T02:22:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f89953d3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8990071900 con 0x7f8990070f20 2023-07-06T02:22:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f89953d3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899015d8e0 con 0x7f8990160000 2023-07-06T02:22:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8990160000 0x7f8990160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8990160000 0x7f8990160420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51284/0 (socket says 172.21.15.172:51284) 2023-07-06T02:22:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 -- 172.21.15.172:0/2914670816 learned_addr learned my addr 172.21.15.172:0/2914670816 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898e7fc700 1 --2- 172.21.15.172:0/2914670816 >> v2:172.21.15.89:3300/0 conn(0x7f8990070f20 0x7f89900713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898dffb700 1 -- 172.21.15.172:0/2914670816 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090522133 0 0) 0x7f89900cb9f0 con 0x7f899015c500 2023-07-06T02:22:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898dffb700 1 -- 172.21.15.172:0/2914670816 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89780035b0 con 0x7f899015c500 2023-07-06T02:22:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 -- 172.21.15.172:0/2914670816 >> v2:172.21.15.89:3300/0 conn(0x7f8990070f20 msgr2=0x7f89900713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 --2- 172.21.15.172:0/2914670816 >> v2:172.21.15.89:3300/0 conn(0x7f8990070f20 0x7f89900713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 -- 172.21.15.172:0/2914670816 >> v1:172.21.15.89:6789/0 conn(0x7f899015c500 legacy=0x7f899006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 -- 172.21.15.172:0/2914670816 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990160960 con 0x7f8990160000 2023-07-06T02:22:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898f7fe700 1 --2- 172.21.15.172:0/2914670816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990160000 0x7f8990160420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f899015db40 tx=0x7f898800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2b368e3f6c3682 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898dffb700 1 -- 172.21.15.172:0/2914670816 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898800e070 con 0x7f8990160000 2023-07-06T02:22:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898dffb700 1 -- 172.21.15.172:0/2914670816 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898800b910 con 0x7f8990160000 2023-07-06T02:22:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.524+0000 7f898dffb700 1 -- 172.21.15.172:0/2914670816 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898800bcc0 con 0x7f8990160000 2023-07-06T02:22:36.532 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 -- 172.21.15.172:0/2914670816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990160000 msgr2=0x7f8990160420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.532 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 --2- 172.21.15.172:0/2914670816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990160000 0x7f8990160420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f899015db40 tx=0x7f898800b330 comp rx=0 tx=0).stop 2023-07-06T02:22:36.532 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 -- 172.21.15.172:0/2914670816 shutdown_connections 2023-07-06T02:22:36.532 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 --2- 172.21.15.172:0/2914670816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990160000 0x7f8990160420 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.533 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 --2- 172.21.15.172:0/2914670816 >> v2:172.21.15.89:3300/0 conn(0x7f8990070f20 0x7f89900713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.533 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 -- 172.21.15.172:0/2914670816 >> 172.21.15.172:0/2914670816 conn(0x7f89900cf1c0 msgr2=0x7f89900cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.533 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 -- 172.21.15.172:0/2914670816 shutdown_connections 2023-07-06T02:22:36.533 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.525+0000 7f89953d3700 1 -- 172.21.15.172:0/2914670816 wait complete. 2023-07-06T02:22:36.533 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.527+0000 7f89953d3700 1 Processor -- start 2023-07-06T02:22:36.533 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.527+0000 7f89953d3700 1 -- start start 2023-07-06T02:22:36.533 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990070f20 0x7f89901f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899015c500 0x7f89901568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8990160000 0x7f899015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89900aecd0 con 0x7f8990160000 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89900aefb0 con 0x7f8990070f20 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89900af290 con 0x7f899015c500 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899015c500 0x7f89901568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899015c500 0x7f89901568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:53196/0 (socket says 172.21.15.172:53196) 2023-07-06T02:22:36.534 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 -- 172.21.15.172:0/3087364011 learned_addr learned my addr 172.21.15.172:0/3087364011 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898effd700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990070f20 0x7f89901f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898f7fe700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8990160000 0x7f899015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 -- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990070f20 msgr2=0x7f89901f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990070f20 0x7f89901f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 -- 172.21.15.172:0/3087364011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8990160000 msgr2=0x7f899015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8990160000 0x7f899015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 -- 172.21.15.172:0/3087364011 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988009b30 con 0x7f899015c500 2023-07-06T02:22:36.535 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f898e7fc700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899015c500 0x7f89901568f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9b0 tx=0x7f898400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.536 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89777fe700 1 -- 172.21.15.172:0/3087364011 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984009710 con 0x7f899015c500 2023-07-06T02:22:36.536 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89777fe700 1 -- 172.21.15.172:0/3087364011 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898400ee50 con 0x7f899015c500 2023-07-06T02:22:36.536 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89900af570 con 0x7f899015c500 2023-07-06T02:22:36.536 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89777fe700 1 -- 172.21.15.172:0/3087364011 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898400ca50 con 0x7f899015c500 2023-07-06T02:22:36.536 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.528+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8990156e30 con 0x7f899015c500 2023-07-06T02:22:36.536 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.529+0000 7f89777fe700 1 -- 172.21.15.172:0/3087364011 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8984005a20 con 0x7f899015c500 2023-07-06T02:22:36.536 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.529+0000 7f89777fe700 1 -- 172.21.15.172:0/3087364011 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f8984006700 con 0x7f899015c500 2023-07-06T02:22:36.537 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.529+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89900afa90 con 0x7f899015c500 2023-07-06T02:22:36.537 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.531+0000 7f89777fe700 1 -- 172.21.15.172:0/3087364011 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f89900afa90 con 0x7f899015c500 2023-07-06T02:22:36.562 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:36.559+0000 7f5b0fb83200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T02:22:36.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.613+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "664b96bb-7c04-4a18-b0ce-d96f5b0fffb5", "id": 6} v 0) v1 -- 0x7f8990074270 con 0x7f899015c500 2023-07-06T02:22:36.623 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.621+0000 7f89777fe700 1 -- 172.21.15.172:0/3087364011 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "664b96bb-7c04-4a18-b0ce-d96f5b0fffb5", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8984021060 con 0x7f899015c500 2023-07-06T02:22:36.624 INFO:teuthology.orchestra.run.smithi172.stdout:6 2023-07-06T02:22:36.625 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899015c500 msgr2=0x7f89901568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.625 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899015c500 0x7f89901568f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9b0 tx=0x7f898400ecc0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.625 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 shutdown_connections 2023-07-06T02:22:36.625 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8990160000 0x7f899015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.625 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8990070f20 0x7f89901f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.626 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 --2- 172.21.15.172:0/3087364011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899015c500 0x7f89901568f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.626 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 >> 172.21.15.172:0/3087364011 conn(0x7f89900cf1c0 msgr2=0x7f899015db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.626 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 shutdown_connections 2023-07-06T02:22:36.626 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.623+0000 7f89953d3700 1 -- 172.21.15.172:0/3087364011 wait complete. 2023-07-06T02:22:36.636 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new fddd98f5-4991-4a63-9400-64b4c37dc8fb 7 2023-07-06T02:22:36.648 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:36.646+0000 7f60e02e5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T02:22:36.717 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:36.714+0000 7f5b0fb83200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T02:22:36.761 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.759+0000 7f52895b1700 1 Processor -- start 2023-07-06T02:22:36.761 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.759+0000 7f52895b1700 1 -- start start 2023-07-06T02:22:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f52895b1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52840cc000 0x7f52840caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f52895b1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528406a990 con 0x7f528406b220 2023-07-06T02:22:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f52895b1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528406ac70 con 0x7f52840b1490 2023-07-06T02:22:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f52895b1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528406d170 con 0x7f52840cc000 2023-07-06T02:22:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f52827fc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52840cc000 0x7f52840caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f5282ffd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f52840b1490 0x7f52840b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56072/0 (socket says 172.21.15.172:56072) 2023-07-06T02:22:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f52827fc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52840cc000 0x7f52840caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51294/0 (socket says 172.21.15.172:51294) 2023-07-06T02:22:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.760+0000 7f5282ffd700 1 -- 172.21.15.172:0/2425736601 learned_addr learned my addr 172.21.15.172:0/2425736601 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-07-06T02:22:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f52827fc700 1 -- 172.21.15.172:0/2425736601 >> v1:172.21.15.172:6789/0 conn(0x7f52840b1490 legacy=0x7f52840b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f52827fc700 1 -- 172.21.15.172:0/2425736601 >> v1:172.21.15.89:6789/0 conn(0x7f528406b220 legacy=0x7f528415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f5281ffb700 1 -- 172.21.15.172:0/2425736601 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482717925 0 0) 0x7f528406ac70 con 0x7f52840b1490 2023-07-06T02:22:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f52827fc700 1 -- 172.21.15.172:0/2425736601 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528406ce60 con 0x7f52840cc000 2023-07-06T02:22:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f52827fc700 1 --2- 172.21.15.172:0/2425736601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840cc000 0x7f52840caeb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5274009840 tx=0x7f5274009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b8e883ce708742 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f5281ffb700 1 -- 172.21.15.172:0/2425736601 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274004230 con 0x7f52840cc000 2023-07-06T02:22:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f5281ffb700 1 -- 172.21.15.172:0/2425736601 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274004390 con 0x7f52840cc000 2023-07-06T02:22:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.761+0000 7f5281ffb700 1 -- 172.21.15.172:0/2425736601 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740036a0 con 0x7f52840cc000 2023-07-06T02:22:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.762+0000 7f52895b1700 1 -- 172.21.15.172:0/2425736601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840cc000 msgr2=0x7f52840caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.762+0000 7f52895b1700 1 --2- 172.21.15.172:0/2425736601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840cc000 0x7f52840caeb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5274009840 tx=0x7f5274009b50 comp rx=0 tx=0).stop 2023-07-06T02:22:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.762+0000 7f52895b1700 1 -- 172.21.15.172:0/2425736601 shutdown_connections 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.762+0000 7f52895b1700 1 --2- 172.21.15.172:0/2425736601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840cc000 0x7f52840caeb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.762+0000 7f52895b1700 1 -- 172.21.15.172:0/2425736601 >> 172.21.15.172:0/2425736601 conn(0x7f52840cc490 msgr2=0x7f52840cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.762+0000 7f52895b1700 1 -- 172.21.15.172:0/2425736601 shutdown_connections 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.762+0000 7f52895b1700 1 -- 172.21.15.172:0/2425736601 wait complete. 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.763+0000 7f52895b1700 1 Processor -- start 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.763+0000 7f52895b1700 1 -- start start 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52895b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f528406b220 0x7f528414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52895b1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840b1490 0x7f5284149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52895b1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52840cc000 0x7f528414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52895b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528414b990 con 0x7f528406b220 2023-07-06T02:22:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52895b1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528414ad10 con 0x7f52840b1490 2023-07-06T02:22:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52895b1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528414aff0 con 0x7f52840cc000 2023-07-06T02:22:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52827fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840b1490 0x7f5284149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52837fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52840cc000 0x7f528414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52837fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52840cc000 0x7f528414a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:53210/0 (socket says 172.21.15.172:53210) 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52827fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840b1490 0x7f5284149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51300/0 (socket says 172.21.15.172:51300) 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52837fe700 1 -- 172.21.15.172:0/3585551457 learned_addr learned my addr 172.21.15.172:0/3585551457 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f5282ffd700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f528406b220 0x7f528414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52827fc700 1 -- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52840cc000 msgr2=0x7f528414a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52827fc700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52840cc000 0x7f528414a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52827fc700 1 -- 172.21.15.172:0/3585551457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f528406b220 msgr2=0x7f528414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52827fc700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f528406b220 0x7f528414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52827fc700 1 -- 172.21.15.172:0/3585551457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740094a0 con 0x7f52840b1490 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.764+0000 7f52837fe700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52840cc000 0x7f528414a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f52827fc700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840b1490 0x7f5284149a00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f52840b2460 tx=0x7f5274004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f526b7fe700 1 -- 172.21.15.172:0/3585551457 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274011630 con 0x7f52840b1490 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f526b7fe700 1 -- 172.21.15.172:0/3585551457 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527401f070 con 0x7f52840b1490 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528414b2d0 con 0x7f52840b1490 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f526b7fe700 1 -- 172.21.15.172:0/3585551457 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740119d0 con 0x7f52840b1490 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5284149f40 con 0x7f52840b1490 2023-07-06T02:22:36.771 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f526b7fe700 1 -- 172.21.15.172:0/3585551457 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5274026480 con 0x7f52840b1490 2023-07-06T02:22:36.772 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.765+0000 7f526b7fe700 1 -- 172.21.15.172:0/3585551457 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f5274021070 con 0x7f52840b1490 2023-07-06T02:22:36.772 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.766+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528414bad0 con 0x7f52840b1490 2023-07-06T02:22:36.772 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.768+0000 7f526b7fe700 1 -- 172.21.15.172:0/3585551457 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f528414bad0 con 0x7f52840b1490 2023-07-06T02:22:36.815 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:36.813+0000 7f60e02e5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T02:22:36.854 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.852+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fddd98f5-4991-4a63-9400-64b4c37dc8fb", "id": 7} v 0) v1 -- 0x7f5284074270 con 0x7f52840b1490 2023-07-06T02:22:36.863 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.861+0000 7f526b7fe700 1 -- 172.21.15.172:0/3585551457 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fddd98f5-4991-4a63-9400-64b4c37dc8fb", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f527402b020 con 0x7f52840b1490 2023-07-06T02:22:36.864 INFO:teuthology.orchestra.run.smithi172.stdout:7 2023-07-06T02:22:36.864 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840b1490 msgr2=0x7f5284149a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840b1490 0x7f5284149a00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f52840b2460 tx=0x7f5274004750 comp rx=0 tx=0).stop 2023-07-06T02:22:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 shutdown_connections 2023-07-06T02:22:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52840cc000 0x7f528414a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f528406b220 0x7f528414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 --2- 172.21.15.172:0/3585551457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52840b1490 0x7f5284149a00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 >> 172.21.15.172:0/3585551457 conn(0x7f52840cc490 msgr2=0x7f5284159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:36.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 shutdown_connections 2023-07-06T02:22:36.866 INFO:teuthology.orchestra.run.smithi172.stderr:2023-07-06T02:22:36.862+0000 7f52895b1700 1 -- 172.21.15.172:0/3585551457 wait complete. 2023-07-06T02:22:36.876 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-06T02:22:36.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-06T02:22:36.879 INFO:tasks.ceph.osd.0:Started 2023-07-06T02:22:36.879 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-06T02:22:36.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-06T02:22:36.881 INFO:tasks.ceph.osd.1:Started 2023-07-06T02:22:36.881 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-06T02:22:36.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-06T02:22:36.883 INFO:tasks.ceph.osd.2:Started 2023-07-06T02:22:36.883 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-06T02:22:36.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-06T02:22:36.885 INFO:tasks.ceph.osd.3:Started 2023-07-06T02:22:36.886 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-06T02:22:36.886 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-06T02:22:36.918 INFO:tasks.ceph.osd.4:Started 2023-07-06T02:22:36.919 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-06T02:22:36.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-06T02:22:36.921 INFO:tasks.ceph.osd.5:Started 2023-07-06T02:22:36.921 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-06T02:22:36.921 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-06T02:22:36.923 INFO:tasks.ceph.osd.6:Started 2023-07-06T02:22:36.924 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-06T02:22:36.924 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-06T02:22:36.926 INFO:tasks.ceph.osd.7:Started 2023-07-06T02:22:36.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:22:37.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.062+0000 7f0aad10f700 1 Processor -- start 2023-07-06T02:22:37.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.063+0000 7f0aad10f700 1 -- start start 2023-07-06T02:22:37.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aad10f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0aa815c690 0x7f0aa80cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aad10f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aad10f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa80cab50 con 0x7f0aa80b19d0 2023-07-06T02:22:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aad10f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa815de50 con 0x7f0aa815c690 2023-07-06T02:22:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aad10f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa815db70 con 0x7f0aa80cbda0 2023-07-06T02:22:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0aa815c690 0x7f0aa80cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0aa815c690 0x7f0aa80cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59598/0 (socket says 172.21.15.89:59598) 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 -- 172.21.15.89:0/3167237572 learned_addr learned my addr 172.21.15.89:0/3167237572 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa659c700 1 --2- 172.21.15.89:0/3167237572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 -- 172.21.15.89:0/3167237572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 msgr2=0x7f0aa80b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 --2- 172.21.15.89:0/3167237572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa80b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 -- 172.21.15.89:0/3167237572 >> v1:172.21.15.172:6789/0 conn(0x7f0aa80b19d0 legacy=0x7f0aa806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 -- 172.21.15.89:0/3167237572 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80b1e30 con 0x7f0aa815c690 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa659c700 1 --2- 172.21.15.89:0/3167237572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa80b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:22:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.064+0000 7f0aa6d9d700 1 --2- 172.21.15.89:0/3167237572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa80cb860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a980098f0 tx=0x7f0a98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67893d9a28334ef6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:37.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aa5d9b700 1 -- 172.21.15.89:0/3167237572 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800b8a0 con 0x7f0aa815c690 2023-07-06T02:22:37.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aa5d9b700 1 -- 172.21.15.89:0/3167237572 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9800ba00 con 0x7f0aa815c690 2023-07-06T02:22:37.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aa5d9b700 1 -- 172.21.15.89:0/3167237572 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800bd90 con 0x7f0aa815c690 2023-07-06T02:22:37.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 -- 172.21.15.89:0/3167237572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 msgr2=0x7f0aa80cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:37.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 --2- 172.21.15.89:0/3167237572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa80cb860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a980098f0 tx=0x7f0a98009c00 comp rx=0 tx=0).stop 2023-07-06T02:22:37.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 -- 172.21.15.89:0/3167237572 shutdown_connections 2023-07-06T02:22:37.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 --2- 172.21.15.89:0/3167237572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa80cb860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 --2- 172.21.15.89:0/3167237572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa80b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 -- 172.21.15.89:0/3167237572 >> 172.21.15.89:0/3167237572 conn(0x7f0aa80cc490 msgr2=0x7f0aa80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:37.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 -- 172.21.15.89:0/3167237572 shutdown_connections 2023-07-06T02:22:37.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.065+0000 7f0aad10f700 1 -- 172.21.15.89:0/3167237572 wait complete. 2023-07-06T02:22:37.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.068+0000 7f0aad10f700 1 Processor -- start 2023-07-06T02:22:37.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.068+0000 7f0aad10f700 1 -- start start 2023-07-06T02:22:37.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aad10f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa80b19d0 0x7f0aa814c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:37.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aad10f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:37.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aad10f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:37.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aad10f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8151410 con 0x7f0aa815c690 2023-07-06T02:22:37.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aad10f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8150790 con 0x7f0aa80b19d0 2023-07-06T02:22:37.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aad10f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8150a70 con 0x7f0aa80cbda0 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa814ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59608/0 (socket says 172.21.15.89:59608) 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa6d9d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa80b19d0 0x7f0aa814c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 -- 172.21.15.89:0/2130327155 learned_addr learned my addr 172.21.15.89:0/2130327155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa659c700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 -- 172.21.15.89:0/2130327155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 msgr2=0x7f0aa814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 -- 172.21.15.89:0/2130327155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa80b19d0 msgr2=0x7f0aa814c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa80b19d0 0x7f0aa814c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 -- 172.21.15.89:0/2130327155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98009550 con 0x7f0aa815c690 2023-07-06T02:22:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.069+0000 7f0aa759e700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa814ac20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000cb50 tx=0x7f0aa000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.070+0000 7f0a8f7fe700 1 -- 172.21.15.89:0/2130327155 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0003c00 con 0x7f0aa815c690 2023-07-06T02:22:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.070+0000 7f0a8f7fe700 1 -- 172.21.15.89:0/2130327155 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0003d60 con 0x7f0aa815c690 2023-07-06T02:22:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.070+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa8150d50 con 0x7f0aa815c690 2023-07-06T02:22:37.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.070+0000 7f0a8f7fe700 1 -- 172.21.15.89:0/2130327155 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000aa20 con 0x7f0aa815c690 2023-07-06T02:22:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.070+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa81504a0 con 0x7f0aa815c690 2023-07-06T02:22:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.070+0000 7f0a8f7fe700 1 -- 172.21.15.89:0/2130327155 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0aa00050d0 con 0x7f0aa815c690 2023-07-06T02:22:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.070+0000 7f0a8f7fe700 1 -- 172.21.15.89:0/2130327155 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0aa0012c80 con 0x7f0aa815c690 2023-07-06T02:22:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.071+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa8151550 con 0x7f0aa815c690 2023-07-06T02:22:37.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.074+0000 7f0a8f7fe700 1 -- 172.21.15.89:0/2130327155 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f0aa0005510 con 0x7f0aa815c690 2023-07-06T02:22:37.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.143+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0aa8074270 con 0x7f0aa815c690 2023-07-06T02:22:37.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.144+0000 7f0a8f7fe700 1 -- 172.21.15.89:0/2130327155 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0aa00056b0 con 0x7f0aa815c690 2023-07-06T02:22:37.147 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:37.147 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":10,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:36.854292+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:22:37.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 msgr2=0x7f0aa814ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:37.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa814ac20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000cb50 tx=0x7f0aa000ce60 comp rx=0 tx=0).stop 2023-07-06T02:22:37.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 shutdown_connections 2023-07-06T02:22:37.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa815c690 0x7f0aa814ac20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa80b19d0 0x7f0aa814c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 --2- 172.21.15.89:0/2130327155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa80cbda0 0x7f0aa814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:37.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 >> 172.21.15.89:0/2130327155 conn(0x7f0aa80cc490 msgr2=0x7f0aa80c61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:37.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 shutdown_connections 2023-07-06T02:22:37.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:37.145+0000 7f0aad10f700 1 -- 172.21.15.89:0/2130327155 wait complete. 2023-07-06T02:22:37.160 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-06T02:22:37.161 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-06T02:22:37.252 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-07-06T02:22:37.250+0000 7fc2861b4640 -1 Falling back to public interface 2023-07-06T02:22:37.454 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:37.451+0000 7f5b0fb83200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T02:22:37.503 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-07-06T02:22:37.500+0000 7f9a06596640 -1 Falling back to public interface 2023-07-06T02:22:37.513 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-07-06T02:22:37.510+0000 7f6b97c86640 -1 Falling back to public interface 2023-07-06T02:22:37.514 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-07-06T02:22:37.511+0000 7f5143cdb640 -1 Falling back to public interface 2023-07-06T02:22:37.526 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-07-06T02:22:37.524+0000 7f1566c81640 -1 Falling back to public interface 2023-07-06T02:22:37.531 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-07-06T02:22:37.529+0000 7f412ea75640 -1 Falling back to public interface 2023-07-06T02:22:37.534 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:37.533+0000 7f60e02e5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T02:22:37.535 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:37.532+0000 7f5b0fb83200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T02:22:37.549 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-07-06T02:22:37.547+0000 7f2de979b640 -1 Falling back to public interface 2023-07-06T02:22:37.550 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-07-06T02:22:37.548+0000 7f58cf99e640 -1 Falling back to public interface 2023-07-06T02:22:37.614 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:37.612+0000 7f60e02e5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T02:22:37.675 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:37.672+0000 7f5b0fb83200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T02:22:37.752 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:37.750+0000 7f60e02e5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T02:22:37.910 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:37.907+0000 7f5b0fb83200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T02:22:37.967 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T02:22:37.985 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:37.982+0000 7f5b0fb83200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T02:22:37.985 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:37.984+0000 7f60e02e5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T02:22:38.059 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:38.057+0000 7f60e02e5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T02:22:38.081 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.078+0000 7ff55c845700 1 Processor -- start 2023-07-06T02:22:38.081 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.078+0000 7ff55c845700 1 -- start start 2023-07-06T02:22:38.083 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.079+0000 7ff55c845700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5580cc950 0x7ff5580cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:38.083 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55c845700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff5580af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:38.083 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55c845700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5580ac6a0 con 0x7ff5580b1060 2023-07-06T02:22:38.084 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55c845700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580ce2b0 con 0x7ff5580cc950 2023-07-06T02:22:38.084 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55c845700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580ac980 con 0x7ff5580ae750 2023-07-06T02:22:38.084 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5580cc950 0x7ff5580cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:38.084 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff555d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff5580b1060 0x7ff5580cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56280/0 (socket says 172.21.15.89:56280) 2023-07-06T02:22:38.084 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff555d9b700 1 -- 172.21.15.89:0/3136845686 learned_addr learned my addr 172.21.15.89:0/3136845686 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:38.084 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff556d9d700 1 --2- 172.21.15.89:0/3136845686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff5580af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:38.084 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55659c700 1 -- 172.21.15.89:0/3136845686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 msgr2=0x7ff5580af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55659c700 1 --2- 172.21.15.89:0/3136845686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff5580af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55659c700 1 -- 172.21.15.89:0/3136845686 >> v1:172.21.15.89:6789/0 conn(0x7ff5580b1060 legacy=0x7ff5580cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55559a700 1 -- 172.21.15.89:0/3136845686 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249725028 0 0) 0x7ff5580ac6a0 con 0x7ff5580b1060 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55659c700 1 -- 172.21.15.89:0/3136845686 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580acc60 con 0x7ff5580cc950 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55659c700 1 --2- 172.21.15.89:0/3136845686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580cc950 0x7ff5580cd660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff548009820 tx=0x7ff548009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d145d5b93b6e6a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55559a700 1 -- 172.21.15.89:0/3136845686 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548005e30 con 0x7ff5580cc950 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55559a700 1 -- 172.21.15.89:0/3136845686 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548006030 con 0x7ff5580cc950 2023-07-06T02:22:38.085 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.080+0000 7ff55559a700 1 -- 172.21.15.89:0/3136845686 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480063c0 con 0x7ff5580cc950 2023-07-06T02:22:38.088 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 -- 172.21.15.89:0/3136845686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580cc950 msgr2=0x7ff5580cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:38.088 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 --2- 172.21.15.89:0/3136845686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580cc950 0x7ff5580cd660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff548009820 tx=0x7ff548009b30 comp rx=0 tx=0).stop 2023-07-06T02:22:38.088 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 -- 172.21.15.89:0/3136845686 shutdown_connections 2023-07-06T02:22:38.088 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 --2- 172.21.15.89:0/3136845686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff5580af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.088 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 --2- 172.21.15.89:0/3136845686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580cc950 0x7ff5580cd660 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.089 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 -- 172.21.15.89:0/3136845686 >> 172.21.15.89:0/3136845686 conn(0x7ff558098da0 msgr2=0x7ff5580cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:38.089 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 -- 172.21.15.89:0/3136845686 shutdown_connections 2023-07-06T02:22:38.089 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.081+0000 7ff55c845700 1 -- 172.21.15.89:0/3136845686 wait complete. 2023-07-06T02:22:38.089 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.083+0000 7ff55c845700 1 Processor -- start 2023-07-06T02:22:38.089 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.083+0000 7ff55c845700 1 -- start start 2023-07-06T02:22:38.089 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff55c845700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff558148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:38.089 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff55c845700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580b1060 0x7ff5580b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff55c845700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5580cc950 0x7ff5580b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff55c845700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55815a550 con 0x7ff5580b1060 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff55c845700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55814e080 con 0x7ff5580cc950 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff55c845700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55814e360 con 0x7ff5580ae750 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff555d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580b1060 0x7ff5580b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff555d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580b1060 0x7ff5580b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59618/0 (socket says 172.21.15.89:59618) 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff55659c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff558148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:38.090 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff556d9d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5580cc950 0x7ff5580b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff555d9b700 1 -- 172.21.15.89:0/4154245884 learned_addr learned my addr 172.21.15.89:0/4154245884 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff555d9b700 1 -- 172.21.15.89:0/4154245884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 msgr2=0x7ff558148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.084+0000 7ff555d9b700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff558148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff555d9b700 1 -- 172.21.15.89:0/4154245884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5580cc950 msgr2=0x7ff5580b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff555d9b700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5580cc950 0x7ff5580b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff555d9b700 1 -- 172.21.15.89:0/4154245884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff548009430 con 0x7ff5580b1060 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff556d9d700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5580cc950 0x7ff5580b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:38.091 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff555d9b700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580b1060 0x7ff5580b28c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0010e0 tx=0x7ff54c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:38.092 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff546ffd700 1 -- 172.21.15.89:0/4154245884 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0074e0 con 0x7ff5580b1060 2023-07-06T02:22:38.092 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff546ffd700 1 -- 172.21.15.89:0/4154245884 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c007640 con 0x7ff5580b1060 2023-07-06T02:22:38.092 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55815a7f0 con 0x7ff5580b1060 2023-07-06T02:22:38.092 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff546ffd700 1 -- 172.21.15.89:0/4154245884 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c005a70 con 0x7ff5580b1060 2023-07-06T02:22:38.092 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55814dca0 con 0x7ff5580b1060 2023-07-06T02:22:38.092 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff546ffd700 1 -- 172.21.15.89:0/4154245884 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff54c01c440 con 0x7ff5580b1060 2023-07-06T02:22:38.092 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.085+0000 7ff546ffd700 1 -- 172.21.15.89:0/4154245884 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff54c012ad0 con 0x7ff5580b1060 2023-07-06T02:22:38.093 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.086+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55815a9e0 con 0x7ff5580b1060 2023-07-06T02:22:38.093 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.088+0000 7ff546ffd700 1 -- 172.21.15.89:0/4154245884 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff54c017070 con 0x7ff5580b1060 2023-07-06T02:22:38.160 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.157+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff558067400 con 0x7ff5580b1060 2023-07-06T02:22:38.160 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.157+0000 7ff546ffd700 1 -- 172.21.15.89:0/4154245884 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff54c012400 con 0x7ff5580b1060 2023-07-06T02:22:38.161 INFO:teuthology.misc.health.smithi089.stdout: 2023-07-06T02:22:38.161 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":10,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:36.854292+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:22:38.162 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580b1060 msgr2=0x7ff5580b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:38.162 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580b1060 0x7ff5580b28c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0010e0 tx=0x7ff54c00ca30 comp rx=0 tx=0).stop 2023-07-06T02:22:38.162 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 shutdown_connections 2023-07-06T02:22:38.162 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5580cc950 0x7ff5580b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.163 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5580ae750 0x7ff558148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.163 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 --2- 172.21.15.89:0/4154245884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5580b1060 0x7ff5580b28c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:38.163 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 >> 172.21.15.89:0/4154245884 conn(0x7ff558098da0 msgr2=0x7ff5580b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:38.163 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 shutdown_connections 2023-07-06T02:22:38.163 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:38.159+0000 7ff55c845700 1 -- 172.21.15.89:0/4154245884 wait complete. 2023-07-06T02:22:38.172 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T02:22:38.265 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:38.262+0000 7f5b0fb83200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T02:22:38.335 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:38.333+0000 7f60e02e5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T02:22:38.484 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:38.481+0000 7f5b0fb83200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T02:22:38.554 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:38.552+0000 7f60e02e5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T02:22:38.640 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:38.637+0000 7f5b0fb83200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T02:22:38.710 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:38.708+0000 7f60e02e5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T02:22:38.754 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:38.751+0000 7f5b0fb83200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T02:22:38.823 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:38.821+0000 7f60e02e5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T02:22:38.904 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:38.901+0000 7f5b0fb83200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T02:22:38.972 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:38.970+0000 7f60e02e5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T02:22:39.180 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:39.177+0000 7f5b0fb83200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T02:22:39.250 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:39.248+0000 7f60e02e5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T02:22:39.364 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:39.361+0000 7f5b0fb83200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T02:22:39.431 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:39.429+0000 7f60e02e5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T02:22:39.954 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:39.951+0000 7f5b0fb83200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T02:22:40.020 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-07-06T02:22:40.018+0000 7f60e02e5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T02:22:40.803 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-07-06T02:22:40.800+0000 7fc2861b4640 -1 osd.0 0 log_to_monitors true 2023-07-06T02:22:40.979 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:40.976+0000 7f5ac7dea700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-06T02:22:40.980 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-07-06T02:22:40.978+0000 7f5ac7dea700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-06T02:22:41.038 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-07-06T02:22:41.035+0000 7f5143cdb640 -1 osd.2 0 log_to_monitors true 2023-07-06T02:22:41.039 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-07-06T02:22:41.035+0000 7f9a06596640 -1 osd.3 0 log_to_monitors true 2023-07-06T02:22:41.093 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-07-06T02:22:41.090+0000 7f6b97c86640 -1 osd.1 0 log_to_monitors true 2023-07-06T02:22:41.117 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-07-06T02:22:41.115+0000 7f1566c81640 -1 osd.4 0 log_to_monitors true 2023-07-06T02:22:41.133 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-07-06T02:22:41.131+0000 7f58cf99e640 -1 osd.6 0 log_to_monitors true 2023-07-06T02:22:41.138 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-07-06T02:22:41.136+0000 7f412ea75640 -1 osd.5 0 log_to_monitors true 2023-07-06T02:22:41.140 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-07-06T02:22:41.139+0000 7f2de979b640 -1 osd.7 0 log_to_monitors true 2023-07-06T02:22:44.180 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-07-06T02:22:44.178+0000 7f6b8c03e700 -1 osd.1 0 waiting for initial osdmap 2023-07-06T02:22:44.191 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-07-06T02:22:44.189+0000 7f155b039700 -1 osd.4 0 waiting for initial osdmap 2023-07-06T02:22:44.207 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-07-06T02:22:44.204+0000 7fc27a56c700 -1 osd.0 0 waiting for initial osdmap 2023-07-06T02:22:44.208 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-07-06T02:22:44.205+0000 7f6b83c18700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.211 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-07-06T02:22:44.209+0000 7f58c3d56700 -1 osd.6 0 waiting for initial osdmap 2023-07-06T02:22:44.217 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-07-06T02:22:44.216+0000 7f1552c13700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.224 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-07-06T02:22:44.222+0000 7fc272146700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.229 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-07-06T02:22:44.227+0000 7f58bb930700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.234 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-07-06T02:22:44.231+0000 7f5138093700 -1 osd.2 0 waiting for initial osdmap 2023-07-06T02:22:44.238 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-07-06T02:22:44.237+0000 7f4122e2d700 -1 osd.5 0 waiting for initial osdmap 2023-07-06T02:22:44.240 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-07-06T02:22:44.238+0000 7f2dddb53700 -1 osd.7 0 waiting for initial osdmap 2023-07-06T02:22:44.246 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-07-06T02:22:44.244+0000 7f512fc6d700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.249 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-07-06T02:22:44.246+0000 7f99fa94e700 -1 osd.3 0 waiting for initial osdmap 2023-07-06T02:22:44.252 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-07-06T02:22:44.251+0000 7f2dd572d700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.256 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-07-06T02:22:44.255+0000 7f411aa07700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.264 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-07-06T02:22:44.261+0000 7f99f2528700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T02:22:44.979 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T02:22:45.093 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.091+0000 7fdfa706f700 1 Processor -- start 2023-07-06T02:22:45.094 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.091+0000 7fdfa706f700 1 -- start start 2023-07-06T02:22:45.096 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.091+0000 7fdfa706f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdfa00aede0 0x7fdfa00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.096 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.091+0000 7fdfa706f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00b0f00 con 0x7fdfa006a770 2023-07-06T02:22:45.096 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.091+0000 7fdfa706f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00caf40 con 0x7fdfa00af940 2023-07-06T02:22:45.097 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.091+0000 7fdfa706f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00af3d0 con 0x7fdfa00aede0 2023-07-06T02:22:45.097 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdfa4e0b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdfa00aede0 0x7fdfa00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.097 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdfa560c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdfa006a770 0x7fdfa00cb420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56392/0 (socket says 172.21.15.89:56392) 2023-07-06T02:22:45.097 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdfa560c700 1 -- 172.21.15.89:0/2533926007 learned_addr learned my addr 172.21.15.89:0/2533926007 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:45.098 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdfa4e0b700 1 -- 172.21.15.89:0/2533926007 >> v1:172.21.15.172:6789/0 conn(0x7fdfa00af940 legacy=0x7fdfa006a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:45.098 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdfa4e0b700 1 -- 172.21.15.89:0/2533926007 >> v1:172.21.15.89:6789/0 conn(0x7fdfa006a770 legacy=0x7fdfa00cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.098 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdf9f7fe700 1 -- 172.21.15.89:0/2533926007 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534464842 0 0) 0x7fdfa00b0f00 con 0x7fdfa006a770 2023-07-06T02:22:45.098 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdfa4e0b700 1 -- 172.21.15.89:0/2533926007 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00b0940 con 0x7fdfa00aede0 2023-07-06T02:22:45.099 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdfa4e0b700 1 --2- 172.21.15.89:0/2533926007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa00d0640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009820 tx=0x7fdf94009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1a60f6b5418b93 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:45.099 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdf9f7fe700 1 -- 172.21.15.89:0/2533926007 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94005e30 con 0x7fdfa00aede0 2023-07-06T02:22:45.099 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdf9f7fe700 1 -- 172.21.15.89:0/2533926007 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf94004030 con 0x7fdfa00aede0 2023-07-06T02:22:45.099 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.092+0000 7fdf9f7fe700 1 -- 172.21.15.89:0/2533926007 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf940043c0 con 0x7fdfa00aede0 2023-07-06T02:22:45.100 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.093+0000 7fdfa706f700 1 -- 172.21.15.89:0/2533926007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 msgr2=0x7fdfa00d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.100 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.093+0000 7fdfa706f700 1 --2- 172.21.15.89:0/2533926007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa00d0640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009820 tx=0x7fdf94009b30 comp rx=0 tx=0).stop 2023-07-06T02:22:45.100 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.093+0000 7fdfa706f700 1 -- 172.21.15.89:0/2533926007 shutdown_connections 2023-07-06T02:22:45.101 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.093+0000 7fdfa706f700 1 --2- 172.21.15.89:0/2533926007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa00d0640 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.101 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.093+0000 7fdfa706f700 1 -- 172.21.15.89:0/2533926007 >> 172.21.15.89:0/2533926007 conn(0x7fdfa0098da0 msgr2=0x7fdfa00cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:45.101 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.093+0000 7fdfa706f700 1 -- 172.21.15.89:0/2533926007 shutdown_connections 2023-07-06T02:22:45.101 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.093+0000 7fdfa706f700 1 -- 172.21.15.89:0/2533926007 wait complete. 2023-07-06T02:22:45.107 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.094+0000 7fdfa706f700 1 Processor -- start 2023-07-06T02:22:45.108 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.094+0000 7fdfa706f700 1 -- start start 2023-07-06T02:22:45.108 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa706f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa006a770 0x7fdfa01ef430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.108 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa706f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa006e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.108 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa706f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa00af940 0x7fdfa00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.108 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa706f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00c8e90 con 0x7fdfa00aede0 2023-07-06T02:22:45.109 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa706f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00c6b60 con 0x7fdfa00af940 2023-07-06T02:22:45.109 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa706f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00c6e20 con 0x7fdfa006a770 2023-07-06T02:22:45.109 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa006e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.110 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa006e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59728/0 (socket says 172.21.15.89:59728) 2023-07-06T02:22:45.110 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 -- 172.21.15.89:0/3015017851 learned_addr learned my addr 172.21.15.89:0/3015017851 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:45.110 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa4e0b700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa006a770 0x7fdfa01ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.111 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdfa560c700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa00af940 0x7fdfa00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.111 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 -- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa006a770 msgr2=0x7fdfa01ef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.111 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa006a770 0x7fdfa01ef430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.111 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 -- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa00af940 msgr2=0x7fdfa00b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.112 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa00af940 0x7fdfa00b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.112 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 -- 172.21.15.89:0/3015017851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94009430 con 0x7fdfa00aede0 2023-07-06T02:22:45.112 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9ffff700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa006e900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00b450 tx=0x7fdf8c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:45.112 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.095+0000 7fdf9d7fa700 1 -- 172.21.15.89:0/3015017851 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00be60 con 0x7fdfa00aede0 2023-07-06T02:22:45.112 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.096+0000 7fdf9d7fa700 1 -- 172.21.15.89:0/3015017851 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c004020 con 0x7fdfa00aede0 2023-07-06T02:22:45.112 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.096+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa00c7100 con 0x7fdfa00aede0 2023-07-06T02:22:45.112 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.096+0000 7fdf9d7fa700 1 -- 172.21.15.89:0/3015017851 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0043d0 con 0x7fdfa00aede0 2023-07-06T02:22:45.113 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.096+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa00c76c0 con 0x7fdfa00aede0 2023-07-06T02:22:45.113 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.097+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa0067400 con 0x7fdfa00aede0 2023-07-06T02:22:45.113 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.097+0000 7fdf9d7fa700 1 -- 172.21.15.89:0/3015017851 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fdf8c0036d0 con 0x7fdfa00aede0 2023-07-06T02:22:45.113 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.097+0000 7fdf9d7fa700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf8406ac50 0x7fdf8406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.113 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.097+0000 7fdfa4e0b700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf8406ac50 0x7fdf8406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.113 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.097+0000 7fdf9d7fa700 1 -- 172.21.15.89:0/3015017851 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fdf8c087b90 con 0x7fdfa00aede0 2023-07-06T02:22:45.113 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.098+0000 7fdfa4e0b700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf8406ac50 0x7fdf8406d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf94000c00 tx=0x7fdf94008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:45.114 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.102+0000 7fdf9d7fa700 1 -- 172.21.15.89:0/3015017851 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf8c053c10 con 0x7fdfa00aede0 2023-07-06T02:22:45.304 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.302+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfa006d530 con 0x7fdfa00aede0 2023-07-06T02:22:45.305 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.302+0000 7fdf9d7fa700 1 -- 172.21.15.89:0/3015017851 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10896 (secure 0 0 0) 0x7fdf8c015070 con 0x7fdfa00aede0 2023-07-06T02:22:45.306 INFO:teuthology.misc.health.smithi089.stdout: 2023-07-06T02:22:45.306 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":13,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:45.218688+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:22:45.307 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf8406ac50 msgr2=0x7fdf8406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.308 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf8406ac50 0x7fdf8406d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf94000c00 tx=0x7fdf94008070 comp rx=0 tx=0).stop 2023-07-06T02:22:45.308 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 msgr2=0x7fdfa006e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.308 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa006e900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00b450 tx=0x7fdf8c00b760 comp rx=0 tx=0).stop 2023-07-06T02:22:45.308 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 shutdown_connections 2023-07-06T02:22:45.308 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa00af940 0x7fdfa00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.308 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa006a770 0x7fdfa01ef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.309 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf8406ac50 0x7fdf8406d100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.309 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 --2- 172.21.15.89:0/3015017851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfa00aede0 0x7fdfa006e900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.309 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 >> 172.21.15.89:0/3015017851 conn(0x7fdfa0098da0 msgr2=0x7fdfa00accf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:45.309 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 shutdown_connections 2023-07-06T02:22:45.309 INFO:teuthology.misc.health.smithi089.stderr:2023-07-06T02:22:45.305+0000 7fdfa706f700 1 -- 172.21.15.89:0/3015017851 wait complete. 2023-07-06T02:22:45.319 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-06T02:22:45.319 INFO:tasks.ceph:Creating RBD pool 2023-07-06T02:22:45.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-06T02:22:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.436+0000 7fa9da9ce700 1 Processor -- start 2023-07-06T02:22:45.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.436+0000 7fa9da9ce700 1 -- start start 2023-07-06T02:22:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9da9ce700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9d415c690 0x7fa9d40cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9da9ce700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40cab50 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9da9ce700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d415db70 con 0x7fa9d40cbda0 2023-07-06T02:22:45.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9da9ce700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d415de50 con 0x7fa9d415c690 2023-07-06T02:22:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9d3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9d415c690 0x7fa9d40cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9d8f6b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa9d40b1ba0 0x7fa9d406bfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56532/0 (socket says 172.21.15.89:56532) 2023-07-06T02:22:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9d8f6b700 1 -- 172.21.15.89:0/1043165278 learned_addr learned my addr 172.21.15.89:0/1043165278 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9d3fff700 1 -- 172.21.15.89:0/1043165278 >> v1:172.21.15.172:6789/0 conn(0x7fa9d40cbda0 legacy=0x7fa9d40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9d3fff700 1 -- 172.21.15.89:0/1043165278 >> v1:172.21.15.89:6789/0 conn(0x7fa9d40b1ba0 legacy=0x7fa9d406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9d2ffd700 1 -- 172.21.15.89:0/1043165278 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540016387 0 0) 0x7fa9d40cab50 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.437+0000 7fa9d3fff700 1 -- 172.21.15.89:0/1043165278 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40cad80 con 0x7fa9d415c690 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9d3fff700 1 --2- 172.21.15.89:0/1043165278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d415c690 0x7fa9d40cb860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009840 tx=0x7fa9c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8ba214524f2280 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9d2ffd700 1 -- 172.21.15.89:0/1043165278 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8004230 con 0x7fa9d415c690 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9d2ffd700 1 -- 172.21.15.89:0/1043165278 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8004390 con 0x7fa9d415c690 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9d2ffd700 1 -- 172.21.15.89:0/1043165278 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80036a0 con 0x7fa9d415c690 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9da9ce700 1 -- 172.21.15.89:0/1043165278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d415c690 msgr2=0x7fa9d40cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/1043165278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d415c690 0x7fa9d40cb860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009840 tx=0x7fa9c8009b50 comp rx=0 tx=0).stop 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9da9ce700 1 -- 172.21.15.89:0/1043165278 shutdown_connections 2023-07-06T02:22:45.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/1043165278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d415c690 0x7fa9d40cb860 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9da9ce700 1 -- 172.21.15.89:0/1043165278 >> 172.21.15.89:0/1043165278 conn(0x7fa9d40cc490 msgr2=0x7fa9d40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9da9ce700 1 -- 172.21.15.89:0/1043165278 shutdown_connections 2023-07-06T02:22:45.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.438+0000 7fa9da9ce700 1 -- 172.21.15.89:0/1043165278 wait complete. 2023-07-06T02:22:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.440+0000 7fa9da9ce700 1 Processor -- start 2023-07-06T02:22:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.440+0000 7fa9da9ce700 1 -- start start 2023-07-06T02:22:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9da9ce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d40b1ba0 0x7fa9d40c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9da9ce700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d40cbda0 0x7fa9d40c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9da9ce700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d415c690 0x7fa9d40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9da9ce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c6ae0 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9da9ce700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c6720 con 0x7fa9d415c690 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9da9ce700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c89f0 con 0x7fa9d40cbda0 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d40b1ba0 0x7fa9d40c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d40b1ba0 0x7fa9d40c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59864/0 (socket says 172.21.15.89:59864) 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 -- 172.21.15.89:0/3847410818 learned_addr learned my addr 172.21.15.89:0/3847410818 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d8f6b700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d415c690 0x7fa9d40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 -- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d40cbda0 msgr2=0x7fa9d40c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d37fe700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d40cbda0 0x7fa9d40c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d40cbda0 0x7fa9d40c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 -- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d415c690 msgr2=0x7fa9d40c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d415c690 0x7fa9d40c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 -- 172.21.15.89:0/3847410818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c80094a0 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d37fe700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d40cbda0 0x7fa9d40c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d8f6b700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d415c690 0x7fa9d40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.441+0000 7fa9d3fff700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d40b1ba0 0x7fa9d40c61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009b90 tx=0x7fa9c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.442+0000 7fa9d0ff9700 1 -- 172.21.15.89:0/3847410818 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8011630 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.442+0000 7fa9d0ff9700 1 -- 172.21.15.89:0/3847410818 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c801e070 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.442+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d40c6cf0 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.442+0000 7fa9d0ff9700 1 -- 172.21.15.89:0/3847410818 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80119b0 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.442+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d40c72b0 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.443+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d40c74a0 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.443+0000 7fa9d0ff9700 1 -- 172.21.15.89:0/3847410818 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fa9c8011b10 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.443+0000 7fa9d0ff9700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa9c006ac50 0x7fa9c006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:45.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.443+0000 7fa9d0ff9700 1 -- 172.21.15.89:0/3847410818 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fa9c808a140 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.443+0000 7fa9d37fe700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa9c006ac50 0x7fa9c006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:45.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.444+0000 7fa9d37fe700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa9c006ac50 0x7fa9c006d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:45.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.448+0000 7fa9d0ff9700 1 -- 172.21.15.89:0/3847410818 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9c8055990 con 0x7fa9d40b1ba0 2023-07-06T02:22:45.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:45.634+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa9d4074270 con 0x7fa9d40b1ba0 2023-07-06T02:22:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.316+0000 7fa9d0ff9700 1 -- 172.21.15.89:0/3847410818 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa9c8058fb0 con 0x7fa9d40b1ba0 2023-07-06T02:22:46.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa9c006ac50 msgr2=0x7fa9c006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:46.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa9c006ac50 0x7fa9c006d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4008040 comp rx=0 tx=0).stop 2023-07-06T02:22:46.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d40b1ba0 msgr2=0x7fa9d40c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d40b1ba0 0x7fa9d40c61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009b90 tx=0x7fa9c8004750 comp rx=0 tx=0).stop 2023-07-06T02:22:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 shutdown_connections 2023-07-06T02:22:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d415c690 0x7fa9d40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa9c006ac50 0x7fa9c006d100 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4008040 comp rx=0 tx=0).stop 2023-07-06T02:22:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d40b1ba0 0x7fa9d40c61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 --2- 172.21.15.89:0/3847410818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d40cbda0 0x7fa9d40c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 >> 172.21.15.89:0/3847410818 conn(0x7fa9d40cc490 msgr2=0x7fa9d415bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 shutdown_connections 2023-07-06T02:22:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.319+0000 7fa9da9ce700 1 -- 172.21.15.89:0/3847410818 wait complete. 2023-07-06T02:22:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'rbd' created 2023-07-06T02:22:46.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-06T02:22:46.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.457+0000 7fd75abe8700 1 Processor -- start 2023-07-06T02:22:46.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.457+0000 7fd75abe8700 1 -- start start 2023-07-06T02:22:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd75abe8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd754071670 0x7fd754071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd75abe8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7540ca0c0 0x7fd7540ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd75abe8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd754072030 con 0x7fd754156c30 2023-07-06T02:22:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd75abe8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7540cdd00 con 0x7fd7540ca0c0 2023-07-06T02:22:46.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd75abe8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd754157000 con 0x7fd754071670 2023-07-06T02:22:46.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd759185700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7540ca0c0 0x7fd7540ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:46.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd759185700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7540ca0c0 0x7fd7540ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59894/0 (socket says 172.21.15.89:59894) 2023-07-06T02:22:46.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd758984700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd754156c30 0x7fd7540cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56560/0 (socket says 172.21.15.89:56560) 2023-07-06T02:22:46.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.458+0000 7fd759185700 1 -- 172.21.15.89:0/113724720 learned_addr learned my addr 172.21.15.89:0/113724720 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:46.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd753fff700 1 --2- 172.21.15.89:0/113724720 >> v2:172.21.15.172:3300/0 conn(0x7fd754071670 0x7fd754071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:46.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd759185700 1 -- 172.21.15.89:0/113724720 >> v2:172.21.15.172:3300/0 conn(0x7fd754071670 msgr2=0x7fd754071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:46.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd759185700 1 --2- 172.21.15.89:0/113724720 >> v2:172.21.15.172:3300/0 conn(0x7fd754071670 0x7fd754071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd759185700 1 -- 172.21.15.89:0/113724720 >> v1:172.21.15.89:6789/0 conn(0x7fd754156c30 legacy=0x7fd7540cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:46.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd759185700 1 -- 172.21.15.89:0/113724720 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7541602c0 con 0x7fd7540ca0c0 2023-07-06T02:22:46.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd7537fe700 1 -- 172.21.15.89:0/113724720 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658815210 0 0) 0x7fd754072030 con 0x7fd754156c30 2023-07-06T02:22:46.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd759185700 1 --2- 172.21.15.89:0/113724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ca4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd75406eaf0 tx=0x7fd74c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97c9fbd03ca56a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:46.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd7537fe700 1 -- 172.21.15.89:0/113724720 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00e070 con 0x7fd7540ca0c0 2023-07-06T02:22:46.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd7537fe700 1 -- 172.21.15.89:0/113724720 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74c00b9c0 con 0x7fd7540ca0c0 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd7537fe700 1 -- 172.21.15.89:0/113724720 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00bd70 con 0x7fd7540ca0c0 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd75abe8700 1 -- 172.21.15.89:0/113724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 msgr2=0x7fd7540ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.459+0000 7fd75abe8700 1 --2- 172.21.15.89:0/113724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ca4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd75406eaf0 tx=0x7fd74c00b330 comp rx=0 tx=0).stop 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.460+0000 7fd75abe8700 1 -- 172.21.15.89:0/113724720 shutdown_connections 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.460+0000 7fd75abe8700 1 --2- 172.21.15.89:0/113724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ca4b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.460+0000 7fd75abe8700 1 --2- 172.21.15.89:0/113724720 >> v2:172.21.15.172:3300/0 conn(0x7fd754071670 0x7fd754071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.460+0000 7fd75abe8700 1 -- 172.21.15.89:0/113724720 >> 172.21.15.89:0/113724720 conn(0x7fd7540cf970 msgr2=0x7fd7540cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:46.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.460+0000 7fd75abe8700 1 -- 172.21.15.89:0/113724720 shutdown_connections 2023-07-06T02:22:46.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.460+0000 7fd75abe8700 1 -- 172.21.15.89:0/113724720 wait complete. 2023-07-06T02:22:46.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.461+0000 7fd75abe8700 1 Processor -- start 2023-07-06T02:22:46.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.461+0000 7fd75abe8700 1 -- start start 2023-07-06T02:22:46.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754071670 0x7fd75414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:46.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:46.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd754156c30 0x7fd7540cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75414d720 con 0x7fd7540ca0c0 2023-07-06T02:22:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75414e540 con 0x7fd754071670 2023-07-06T02:22:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75414d930 con 0x7fd754156c30 2023-07-06T02:22:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59898/0 (socket says 172.21.15.89:59898) 2023-07-06T02:22:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 -- 172.21.15.89:0/2021930207 learned_addr learned my addr 172.21.15.89:0/2021930207 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:46.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd758984700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754071670 0x7fd75414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd759185700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd754156c30 0x7fd7540cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 -- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd754156c30 msgr2=0x7fd7540cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd754156c30 0x7fd7540cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 -- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754071670 msgr2=0x7fd75414cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754071670 0x7fd75414cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:46.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 -- 172.21.15.89:0/2021930207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c009b30 con 0x7fd7540ca0c0 2023-07-06T02:22:46.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd759185700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd754156c30 0x7fd7540cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:22:46.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd753fff700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ae0e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd74000e810 tx=0x7fd74000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:46.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd7517fa700 1 -- 172.21.15.89:0/2021930207 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740013610 con 0x7fd7540ca0c0 2023-07-06T02:22:46.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd7517fa700 1 -- 172.21.15.89:0/2021930207 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd740013770 con 0x7fd7540ca0c0 2023-07-06T02:22:46.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75414dc10 con 0x7fd7540ca0c0 2023-07-06T02:22:46.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd7517fa700 1 -- 172.21.15.89:0/2021930207 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740013b20 con 0x7fd7540ca0c0 2023-07-06T02:22:46.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.462+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75414e1d0 con 0x7fd7540ca0c0 2023-07-06T02:22:46.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.463+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7540742e0 con 0x7fd7540ca0c0 2023-07-06T02:22:46.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.464+0000 7fd7517fa700 1 -- 172.21.15.89:0/2021930207 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fd740013c80 con 0x7fd7540ca0c0 2023-07-06T02:22:46.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.464+0000 7fd7517fa700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd73806ac50 0x7fd73806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:46.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.464+0000 7fd7517fa700 1 -- 172.21.15.89:0/2021930207 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7fd740088c00 con 0x7fd7540ca0c0 2023-07-06T02:22:46.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.464+0000 7fd758984700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd73806ac50 0x7fd73806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:46.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.464+0000 7fd758984700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd73806ac50 0x7fd73806d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd748000f60 tx=0x7fd748007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:46.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.469+0000 7fd7517fa700 1 -- 172.21.15.89:0/2021930207 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd740054100 con 0x7fd7540ca0c0 2023-07-06T02:22:46.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:46.659+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd75414e390 con 0x7fd7540ca0c0 2023-07-06T02:22:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.319+0000 7fd7517fa700 1 -- 172.21.15.89:0/2021930207 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd740057720 con 0x7fd7540ca0c0 2023-07-06T02:22:47.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd73806ac50 msgr2=0x7fd73806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd73806ac50 0x7fd73806d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd748000f60 tx=0x7fd748007e70 comp rx=0 tx=0).stop 2023-07-06T02:22:47.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 msgr2=0x7fd7540ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ae0e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd74000e810 tx=0x7fd74000eb20 comp rx=0 tx=0).stop 2023-07-06T02:22:47.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 shutdown_connections 2023-07-06T02:22:47.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd754156c30 0x7fd7540cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754071670 0x7fd75414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd73806ac50 0x7fd73806d100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 --2- 172.21.15.89:0/2021930207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7540ca0c0 0x7fd7540ae0e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 >> 172.21.15.89:0/2021930207 conn(0x7fd7540cf970 msgr2=0x7fd754071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:47.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.323+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 shutdown_connections 2023-07-06T02:22:47.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.324+0000 7fd75abe8700 1 -- 172.21.15.89:0/2021930207 wait complete. 2023-07-06T02:22:47.328 INFO:teuthology.orchestra.run.smithi089.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-06T02:22:47.341 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-06T02:22:47.342 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T02:22:47.342 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T02:22:47.344 INFO:tasks.ceph.mds.a:Started 2023-07-06T02:22:47.344 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T02:22:47.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T02:22:47.346 INFO:tasks.ceph.mds.c:Started 2023-07-06T02:22:47.347 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T02:22:47.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T02:22:47.349 INFO:tasks.ceph.mds.b:Started 2023-07-06T02:22:47.350 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T02:22:47.350 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T02:22:47.352 INFO:tasks.ceph.mds.d:Started 2023-07-06T02:22:47.352 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-06T02:22:47.353 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:22:47.437 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-07-06T02:22:47.441 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-07-06T02:22:47.442 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-07-06T02:22:47.442 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-07-06T02:22:47.442 INFO:tasks.ceph.mds.d.smithi172.stderr:2023-07-06T02:22:47.440+0000 7fdbca134700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-07-06T02:22:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.494+0000 7f596b6e9700 1 Processor -- start 2023-07-06T02:22:47.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.494+0000 7f596b6e9700 1 -- start start 2023-07-06T02:22:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f596b6e9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59640b1ba0 0x7f5964167340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f596b6e9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640cb3d0 con 0x7f59641600b0 2023-07-06T02:22:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f596b6e9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5964156e80 con 0x7f5964156ab0 2023-07-06T02:22:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f596b6e9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640ca370 con 0x7f59640b1ba0 2023-07-06T02:22:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f5968c84700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f59641600b0 0x7f59640b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56618/0 (socket says 172.21.15.89:56618) 2023-07-06T02:22:47.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f5968c84700 1 -- 172.21.15.89:0/3471883539 learned_addr learned my addr 172.21.15.89:0/3471883539 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f5969c86700 1 --2- 172.21.15.89:0/3471883539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59640b1ba0 0x7f5964167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.495+0000 7f595bfff700 1 -- 172.21.15.89:0/3471883539 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993190514 0 0) 0x7f59640cb3d0 con 0x7f59641600b0 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f595bfff700 1 -- 172.21.15.89:0/3471883539 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59500035b0 con 0x7f59641600b0 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f595bfff700 1 -- 172.21.15.89:0/3471883539 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880436961 0 0) 0x7f5964156e80 con 0x7f5964156ab0 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f595bfff700 1 -- 172.21.15.89:0/3471883539 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59640cb3d0 con 0x7f5964156ab0 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f5969c86700 1 -- 172.21.15.89:0/3471883539 >> v1:172.21.15.172:6789/0 conn(0x7f5964156ab0 legacy=0x7f5964160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f5969c86700 1 -- 172.21.15.89:0/3471883539 >> v1:172.21.15.89:6789/0 conn(0x7f59641600b0 legacy=0x7f59640b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f5969c86700 1 -- 172.21.15.89:0/3471883539 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964160d80 con 0x7f59640b1ba0 2023-07-06T02:22:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f5969c86700 1 --2- 172.21.15.89:0/3471883539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59640b1ba0 0x7f5964167340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f595c009dc0 tx=0x7f595c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83bd99551a47243f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f595bfff700 1 -- 172.21.15.89:0/3471883539 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c00e070 con 0x7f59640b1ba0 2023-07-06T02:22:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f595bfff700 1 -- 172.21.15.89:0/3471883539 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c00bb20 con 0x7f59640b1ba0 2023-07-06T02:22:47.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.496+0000 7f595bfff700 1 -- 172.21.15.89:0/3471883539 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c012430 con 0x7f59640b1ba0 2023-07-06T02:22:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.497+0000 7f596b6e9700 1 -- 172.21.15.89:0/3471883539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59640b1ba0 msgr2=0x7f5964167340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.497+0000 7f596b6e9700 1 --2- 172.21.15.89:0/3471883539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59640b1ba0 0x7f5964167340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f595c009dc0 tx=0x7f595c00b330 comp rx=0 tx=0).stop 2023-07-06T02:22:47.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.497+0000 7f596b6e9700 1 -- 172.21.15.89:0/3471883539 shutdown_connections 2023-07-06T02:22:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.497+0000 7f596b6e9700 1 --2- 172.21.15.89:0/3471883539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59640b1ba0 0x7f5964167340 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.497+0000 7f596b6e9700 1 -- 172.21.15.89:0/3471883539 >> 172.21.15.89:0/3471883539 conn(0x7f59640cc490 msgr2=0x7f59640cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.497+0000 7f596b6e9700 1 -- 172.21.15.89:0/3471883539 shutdown_connections 2023-07-06T02:22:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.497+0000 7f596b6e9700 1 -- 172.21.15.89:0/3471883539 wait complete. 2023-07-06T02:22:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.498+0000 7f596b6e9700 1 Processor -- start 2023-07-06T02:22:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.498+0000 7f596b6e9700 1 -- start start 2023-07-06T02:22:47.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f596b6e9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59640b1ba0 0x7f59641e8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f596b6e9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5964156ab0 0x7f59640c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f596b6e9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59641600b0 0x7f59640c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f596b6e9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59641e7cd0 con 0x7f5964156ab0 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f596b6e9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640c35e0 con 0x7f59640b1ba0 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f596b6e9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640c7610 con 0x7f59641600b0 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f5968c84700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5964156ab0 0x7f59640c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f5968c84700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5964156ab0 0x7f59640c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59956/0 (socket says 172.21.15.89:59956) 2023-07-06T02:22:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f5968c84700 1 -- 172.21.15.89:0/1237155197 learned_addr learned my addr 172.21.15.89:0/1237155197 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f5969c86700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59641600b0 0x7f59640c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.499+0000 7f5969485700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59640b1ba0 0x7f59641e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5968c84700 1 -- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59641600b0 msgr2=0x7f59640c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5968c84700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59641600b0 0x7f59640c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5968c84700 1 -- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59640b1ba0 msgr2=0x7f59641e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5968c84700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59640b1ba0 0x7f59641e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5968c84700 1 -- 172.21.15.89:0/1237155197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c009a70 con 0x7f5964156ab0 2023-07-06T02:22:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5968c84700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5964156ab0 0x7f59640c6b90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5960001eb0 tx=0x7f596000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5959ffb700 1 -- 172.21.15.89:0/1237155197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960005d70 con 0x7f5964156ab0 2023-07-06T02:22:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5959ffb700 1 -- 172.21.15.89:0/1237155197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59600070d0 con 0x7f5964156ab0 2023-07-06T02:22:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640c3880 con 0x7f5964156ab0 2023-07-06T02:22:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f5959ffb700 1 -- 172.21.15.89:0/1237155197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960007460 con 0x7f5964156ab0 2023-07-06T02:22:47.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.500+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59640c3b60 con 0x7f5964156ab0 2023-07-06T02:22:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.501+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59640d0730 con 0x7f5964156ab0 2023-07-06T02:22:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.502+0000 7f5959ffb700 1 -- 172.21.15.89:0/1237155197 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f59600075c0 con 0x7f5964156ab0 2023-07-06T02:22:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.502+0000 7f5959ffb700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f594806ac50 0x7f594806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.502+0000 7f5959ffb700 1 -- 172.21.15.89:0/1237155197 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f59600888d0 con 0x7f5964156ab0 2023-07-06T02:22:47.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.502+0000 7f5969485700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f594806ac50 0x7f594806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.503+0000 7f5969485700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f594806ac50 0x7f594806d100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f59540011e0 tx=0x7f5954009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:47.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.506+0000 7f5959ffb700 1 -- 172.21.15.89:0/1237155197 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5960053d90 con 0x7f5964156ab0 2023-07-06T02:22:47.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.700+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59640cf330 con 0x7f5964156ab0 2023-07-06T02:22:47.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.701+0000 7f5959ffb700 1 -- 172.21.15.89:0/1237155197 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14593 (secure 0 0 0) 0x7f5960088070 con 0x7f5964156ab0 2023-07-06T02:22:47.704 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:47.705 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":15,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:47.316074+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:22:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f594806ac50 msgr2=0x7f594806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f594806ac50 0x7f594806d100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f59540011e0 tx=0x7f5954009040 comp rx=0 tx=0).stop 2023-07-06T02:22:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5964156ab0 msgr2=0x7f59640c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5964156ab0 0x7f59640c6b90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5960001eb0 tx=0x7f596000c7b0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 shutdown_connections 2023-07-06T02:22:47.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59641600b0 0x7f59640c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59640b1ba0 0x7f59641e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f594806ac50 0x7f594806d100 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 --2- 172.21.15.89:0/1237155197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5964156ab0 0x7f59640c6b90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 >> 172.21.15.89:0/1237155197 conn(0x7f59640cc490 msgr2=0x7f59640b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:47.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 shutdown_connections 2023-07-06T02:22:47.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.703+0000 7f596b6e9700 1 -- 172.21.15.89:0/1237155197 wait complete. 2023-07-06T02:22:47.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T02:22:47.721 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:22:47.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.848+0000 7fd3ba2a2700 1 Processor -- start 2023-07-06T02:22:47.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.848+0000 7fd3ba2a2700 1 -- start start 2023-07-06T02:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3ba2a2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd3b4071630 0x7fd3b40caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3ba2a2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd3b406bcd0 0x7fd3b415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3ba2a2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b406a500 con 0x7fd3b415a3e0 2023-07-06T02:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3ba2a2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b406ec60 con 0x7fd3b4071630 2023-07-06T02:22:47.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3ba2a2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b406f190 con 0x7fd3b406bcd0 2023-07-06T02:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd3b4071630 0x7fd3b40caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd3b415a3e0 0x7fd3b4070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56636/0 (socket says 172.21.15.89:56636) 2023-07-06T02:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b3fff700 1 -- 172.21.15.89:0/2578493870 learned_addr learned my addr 172.21.15.89:0/2578493870 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b883f700 1 --2- 172.21.15.89:0/2578493870 >> v2:172.21.15.172:3300/0 conn(0x7fd3b406bcd0 0x7fd3b415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b37fe700 1 -- 172.21.15.89:0/2578493870 >> v2:172.21.15.172:3300/0 conn(0x7fd3b406bcd0 msgr2=0x7fd3b415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:47.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b37fe700 1 --2- 172.21.15.89:0/2578493870 >> v2:172.21.15.172:3300/0 conn(0x7fd3b406bcd0 0x7fd3b415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b37fe700 1 -- 172.21.15.89:0/2578493870 >> v1:172.21.15.89:6789/0 conn(0x7fd3b415a3e0 legacy=0x7fd3b4070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.849+0000 7fd3b37fe700 1 -- 172.21.15.89:0/2578493870 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b40cb980 con 0x7fd3b4071630 2023-07-06T02:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3b37fe700 1 --2- 172.21.15.89:0/2578493870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b4071630 0x7fd3b40caf80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4066cf0 tx=0x7fd3a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561ed2adaab94f3c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3b2ffd700 1 -- 172.21.15.89:0/2578493870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8011830 con 0x7fd3b4071630 2023-07-06T02:22:47.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3b2ffd700 1 -- 172.21.15.89:0/2578493870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a8011990 con 0x7fd3b4071630 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3b2ffd700 1 -- 172.21.15.89:0/2578493870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8011d20 con 0x7fd3b4071630 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/2578493870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b4071630 msgr2=0x7fd3b40caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/2578493870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b4071630 0x7fd3b40caf80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4066cf0 tx=0x7fd3a8009b20 comp rx=0 tx=0).stop 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/2578493870 shutdown_connections 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/2578493870 >> v2:172.21.15.172:3300/0 conn(0x7fd3b406bcd0 0x7fd3b415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/2578493870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b4071630 0x7fd3b40caf80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.850+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/2578493870 >> 172.21.15.89:0/2578493870 conn(0x7fd3b40cf1c0 msgr2=0x7fd3b40cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:47.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.851+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/2578493870 shutdown_connections 2023-07-06T02:22:47.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.851+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/2578493870 wait complete. 2023-07-06T02:22:47.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.852+0000 7fd3ba2a2700 1 Processor -- start 2023-07-06T02:22:47.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.852+0000 7fd3ba2a2700 1 -- start start 2023-07-06T02:22:47.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3b406bcd0 0x7fd3b41464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3b4071630 0x7fd3b40c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b415a3e0 0x7fd3b415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c4850 con 0x7fd3b415a3e0 2023-07-06T02:22:47.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c5b90 con 0x7fd3b406bcd0 2023-07-06T02:22:47.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c4af0 con 0x7fd3b4071630 2023-07-06T02:22:47.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b415a3e0 0x7fd3b415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b415a3e0 0x7fd3b415bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59976/0 (socket says 172.21.15.89:59976) 2023-07-06T02:22:47.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 -- 172.21.15.89:0/3953074179 learned_addr learned my addr 172.21.15.89:0/3953074179 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:47.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b3fff700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3b406bcd0 0x7fd3b41464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b37fe700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3b4071630 0x7fd3b40c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 -- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3b4071630 msgr2=0x7fd3b40c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3b4071630 0x7fd3b40c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 -- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3b406bcd0 msgr2=0x7fd3b41464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:47.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3b406bcd0 0x7fd3b41464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:47.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 -- 172.21.15.89:0/3953074179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a80094a0 con 0x7fd3b415a3e0 2023-07-06T02:22:47.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b883f700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b415a3e0 0x7fd3b415bab0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac007e60 tx=0x7fd3ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b0ff9700 1 -- 172.21.15.89:0/3953074179 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00ed30 con 0x7fd3b415a3e0 2023-07-06T02:22:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b0ff9700 1 -- 172.21.15.89:0/3953074179 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac014420 con 0x7fd3b415a3e0 2023-07-06T02:22:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b40c4dd0 con 0x7fd3b415a3e0 2023-07-06T02:22:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3b0ff9700 1 -- 172.21.15.89:0/3953074179 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0147d0 con 0x7fd3b415a3e0 2023-07-06T02:22:47.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.853+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b40c5390 con 0x7fd3b415a3e0 2023-07-06T02:22:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.855+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b4074270 con 0x7fd3b415a3e0 2023-07-06T02:22:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.855+0000 7fd3b0ff9700 1 -- 172.21.15.89:0/3953074179 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fd3ac014930 con 0x7fd3b415a3e0 2023-07-06T02:22:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.855+0000 7fd3b0ff9700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd3a006ac50 0x7fd3a006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.855+0000 7fd3b3fff700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd3a006ac50 0x7fd3a006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.855+0000 7fd3b0ff9700 1 -- 172.21.15.89:0/3953074179 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd3ac0896a0 con 0x7fd3b415a3e0 2023-07-06T02:22:47.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.856+0000 7fd3b3fff700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd3a006ac50 0x7fd3a006d100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005910 tx=0x7fd3a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:47.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:47.860+0000 7fd3b0ff9700 1 -- 172.21.15.89:0/3953074179 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd3ac054c30 con 0x7fd3b415a3e0 2023-07-06T02:22:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.054+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3b4159720 con 0x7fd3b415a3e0 2023-07-06T02:22:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.055+0000 7fd3b0ff9700 1 -- 172.21.15.89:0/3953074179 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd3ac058250 con 0x7fd3b415a3e0 2023-07-06T02:22:48.059 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:22:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd3a006ac50 msgr2=0x7fd3a006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd3a006ac50 0x7fd3a006d100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005910 tx=0x7fd3a4009040 comp rx=0 tx=0).stop 2023-07-06T02:22:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b415a3e0 msgr2=0x7fd3b415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b415a3e0 0x7fd3b415bab0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac007e60 tx=0x7fd3ac00e3c0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 shutdown_connections 2023-07-06T02:22:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd3a006ac50 0x7fd3a006d100 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3b415a3e0 0x7fd3b415bab0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3b406bcd0 0x7fd3b41464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 --2- 172.21.15.89:0/3953074179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3b4071630 0x7fd3b40c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 >> 172.21.15.89:0/3953074179 conn(0x7fd3b40cf1c0 msgr2=0x7fd3b40cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 shutdown_connections 2023-07-06T02:22:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.057+0000 7fd3ba2a2700 1 -- 172.21.15.89:0/3953074179 wait complete. 2023-07-06T02:22:48.072 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:22:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.193+0000 7f350a56d700 1 Processor -- start 2023-07-06T02:22:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.193+0000 7f350a56d700 1 -- start start 2023-07-06T02:22:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f350a56d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f350415c610 0x7f35040cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f350a56d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3504070a40 0x7f35040cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f350a56d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35040cb150 con 0x7f35040b5e50 2023-07-06T02:22:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f350a56d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35040cb430 con 0x7f3504070a40 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f350a56d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35040b6310 con 0x7f350415c610 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3508b0a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3504070a40 0x7f35040cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f35037fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f35040b5e50 0x7f3504070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56682/0 (socket says 172.21.15.89:56682) 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3508b0a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3504070a40 0x7f35040cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60016/0 (socket says 172.21.15.89:60016) 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f35037fe700 1 -- 172.21.15.89:0/1617494663 learned_addr learned my addr 172.21.15.89:0/1617494663 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3503fff700 1 --2- 172.21.15.89:0/1617494663 >> v2:172.21.15.172:3300/0 conn(0x7f350415c610 0x7f35040cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3508b0a700 1 -- 172.21.15.89:0/1617494663 >> v2:172.21.15.172:3300/0 conn(0x7f350415c610 msgr2=0x7f35040cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3508b0a700 1 --2- 172.21.15.89:0/1617494663 >> v2:172.21.15.172:3300/0 conn(0x7f350415c610 0x7f35040cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3508b0a700 1 -- 172.21.15.89:0/1617494663 >> v1:172.21.15.89:6789/0 conn(0x7f35040b5e50 legacy=0x7f3504070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3508b0a700 1 -- 172.21.15.89:0/1617494663 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35040b65f0 con 0x7f3504070a40 2023-07-06T02:22:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3502ffd700 1 -- 172.21.15.89:0/1617494663 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137555673 0 0) 0x7f35040cb150 con 0x7f35040b5e50 2023-07-06T02:22:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3508b0a700 1 --2- 172.21.15.89:0/1617494663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f35040cb8c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35040d1680 tx=0x7f34fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63261fe72b85725 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3502ffd700 1 -- 172.21.15.89:0/1617494663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00e070 con 0x7f3504070a40 2023-07-06T02:22:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3502ffd700 1 -- 172.21.15.89:0/1617494663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc00bb20 con 0x7f3504070a40 2023-07-06T02:22:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.194+0000 7f3502ffd700 1 -- 172.21.15.89:0/1617494663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc012430 con 0x7f3504070a40 2023-07-06T02:22:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 -- 172.21.15.89:0/1617494663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 msgr2=0x7f35040cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 --2- 172.21.15.89:0/1617494663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f35040cb8c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35040d1680 tx=0x7f34fc00b330 comp rx=0 tx=0).stop 2023-07-06T02:22:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 -- 172.21.15.89:0/1617494663 shutdown_connections 2023-07-06T02:22:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 --2- 172.21.15.89:0/1617494663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f35040cb8c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 --2- 172.21.15.89:0/1617494663 >> v2:172.21.15.172:3300/0 conn(0x7f350415c610 0x7f35040cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 -- 172.21.15.89:0/1617494663 >> 172.21.15.89:0/1617494663 conn(0x7f35040d0b00 msgr2=0x7f35040d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:48.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 -- 172.21.15.89:0/1617494663 shutdown_connections 2023-07-06T02:22:48.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.195+0000 7f350a56d700 1 -- 172.21.15.89:0/1617494663 wait complete. 2023-07-06T02:22:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.197+0000 7f350a56d700 1 Processor -- start 2023-07-06T02:22:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.197+0000 7f350a56d700 1 -- start start 2023-07-06T02:22:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f350415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35040b5e50 0x7f35041f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f350415c610 0x7f3504155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35041562d0 con 0x7f3504070a40 2023-07-06T02:22:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f350415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35041565b0 con 0x7f350415c610 2023-07-06T02:22:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3504156890 con 0x7f35040b5e50 2023-07-06T02:22:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f350415f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60018/0 (socket says 172.21.15.89:60018) 2023-07-06T02:22:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 -- 172.21.15.89:0/1397543411 learned_addr learned my addr 172.21.15.89:0/1397543411 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f35037fe700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35040b5e50 0x7f35041f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3508b0a700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f350415c610 0x7f3504155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 -- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35040b5e50 msgr2=0x7f35041f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35040b5e50 0x7f35041f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 -- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f350415c610 msgr2=0x7f3504155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f350415c610 0x7f3504155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 -- 172.21.15.89:0/1397543411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc009a70 con 0x7f3504070a40 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3503fff700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f350415f340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34f800e870 tx=0x7f34f800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3500ff9700 1 -- 172.21.15.89:0/1397543411 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8009590 con 0x7f3504070a40 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3500ff9700 1 -- 172.21.15.89:0/1397543411 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f8013680 con 0x7f3504070a40 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f3500ff9700 1 -- 172.21.15.89:0/1397543411 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80139f0 con 0x7f3504070a40 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3504156b70 con 0x7f3504070a40 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.198+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35041f1670 con 0x7f3504070a40 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.199+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3504074270 con 0x7f3504070a40 2023-07-06T02:22:48.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.200+0000 7f3500ff9700 1 -- 172.21.15.89:0/1397543411 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f34f8013b50 con 0x7f3504070a40 2023-07-06T02:22:48.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.200+0000 7f3500ff9700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f34f006ac50 0x7f34f006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.200+0000 7f3500ff9700 1 -- 172.21.15.89:0/1397543411 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f34f8088e50 con 0x7f3504070a40 2023-07-06T02:22:48.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.200+0000 7f35037fe700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f34f006ac50 0x7f34f006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.201+0000 7f35037fe700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f34f006ac50 0x7f34f006d100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f35040cf4c0 tx=0x7f34f4007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.205+0000 7f3500ff9700 1 -- 172.21.15.89:0/1397543411 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f34f80543c0 con 0x7f3504070a40 2023-07-06T02:22:48.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.398+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35041593e0 con 0x7f3504070a40 2023-07-06T02:22:48.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.398+0000 7f3500ff9700 1 -- 172.21.15.89:0/1397543411 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34f80170a0 con 0x7f3504070a40 2023-07-06T02:22:48.402 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:22:48.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f34f006ac50 msgr2=0x7f34f006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f34f006ac50 0x7f34f006d100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f35040cf4c0 tx=0x7f34f4007de0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 msgr2=0x7f350415f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f350415f340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34f800e870 tx=0x7f34f800eb80 comp rx=0 tx=0).stop 2023-07-06T02:22:48.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 shutdown_connections 2023-07-06T02:22:48.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f350415c610 0x7f3504155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f34f006ac50 0x7f34f006d100 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3504070a40 0x7f350415f340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 --2- 172.21.15.89:0/1397543411 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35040b5e50 0x7f35041f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 >> 172.21.15.89:0/1397543411 conn(0x7f35040d0b00 msgr2=0x7f35040b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:48.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 shutdown_connections 2023-07-06T02:22:48.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.400+0000 7f350a56d700 1 -- 172.21.15.89:0/1397543411 wait complete. 2023-07-06T02:22:48.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:22:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.536+0000 7f612b1ab700 1 Processor -- start 2023-07-06T02:22:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.536+0000 7f612b1ab700 1 -- start start 2023-07-06T02:22:48.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.537+0000 7f612b1ab700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6124070a40 0x7f6124167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.537+0000 7f612b1ab700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61240cb150 con 0x7f61240b5e50 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.537+0000 7f612b1ab700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61240cb430 con 0x7f612415c610 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.537+0000 7f612b1ab700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61240d05e0 con 0x7f6124070a40 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.537+0000 7f6129748700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6124070a40 0x7f6124167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.537+0000 7f6129748700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6124070a40 0x7f6124167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60042/0 (socket says 172.21.15.89:60042) 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.537+0000 7f6129748700 1 -- 172.21.15.89:0/3801186600 learned_addr learned my addr 172.21.15.89:0/3801186600 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f6129748700 1 -- 172.21.15.89:0/3801186600 >> v1:172.21.15.172:6789/0 conn(0x7f612415c610 legacy=0x7f61240cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f6129748700 1 -- 172.21.15.89:0/3801186600 >> v1:172.21.15.89:6789/0 conn(0x7f61240b5e50 legacy=0x7f6124070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f6129748700 1 -- 172.21.15.89:0/3801186600 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61240b6310 con 0x7f6124070a40 2023-07-06T02:22:48.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f61237fe700 1 -- 172.21.15.89:0/3801186600 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446813908 0 0) 0x7f61240cb430 con 0x7f612415c610 2023-07-06T02:22:48.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f6129748700 1 --2- 172.21.15.89:0/3801186600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f6124167050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f611c009e50 tx=0x7f611c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe8cb41f1505504 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f61237fe700 1 -- 172.21.15.89:0/3801186600 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c00e070 con 0x7f6124070a40 2023-07-06T02:22:48.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f61237fe700 1 -- 172.21.15.89:0/3801186600 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c003a20 con 0x7f6124070a40 2023-07-06T02:22:48.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.538+0000 7f61237fe700 1 -- 172.21.15.89:0/3801186600 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c003dd0 con 0x7f6124070a40 2023-07-06T02:22:48.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.539+0000 7f612b1ab700 1 -- 172.21.15.89:0/3801186600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 msgr2=0x7f6124167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.539+0000 7f612b1ab700 1 --2- 172.21.15.89:0/3801186600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f6124167050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f611c009e50 tx=0x7f611c00b330 comp rx=0 tx=0).stop 2023-07-06T02:22:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.539+0000 7f612b1ab700 1 -- 172.21.15.89:0/3801186600 shutdown_connections 2023-07-06T02:22:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.539+0000 7f612b1ab700 1 --2- 172.21.15.89:0/3801186600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f6124167050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.539+0000 7f612b1ab700 1 -- 172.21.15.89:0/3801186600 >> 172.21.15.89:0/3801186600 conn(0x7f61240d0b00 msgr2=0x7f61240d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.539+0000 7f612b1ab700 1 -- 172.21.15.89:0/3801186600 shutdown_connections 2023-07-06T02:22:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.539+0000 7f612b1ab700 1 -- 172.21.15.89:0/3801186600 wait complete. 2023-07-06T02:22:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.541+0000 7f612b1ab700 1 Processor -- start 2023-07-06T02:22:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.541+0000 7f612b1ab700 1 -- start start 2023-07-06T02:22:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f612b1ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f612415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f612b1ab700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61240b5e50 0x7f61240ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f612b1ab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f612415c610 0x7f61240c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f612b1ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61240c8710 con 0x7f6124070a40 2023-07-06T02:22:48.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f612b1ab700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612415ec30 con 0x7f61240b5e50 2023-07-06T02:22:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f612b1ab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61240c8430 con 0x7f612415c610 2023-07-06T02:22:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f612415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f612415f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60052/0 (socket says 172.21.15.89:60052) 2023-07-06T02:22:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 -- 172.21.15.89:0/654519412 learned_addr learned my addr 172.21.15.89:0/654519412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6129748700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f612415c610 0x7f61240c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6123fff700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61240b5e50 0x7f61240ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 -- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f612415c610 msgr2=0x7f61240c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f612415c610 0x7f61240c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 -- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61240b5e50 msgr2=0x7f61240ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61240b5e50 0x7f61240ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.542+0000 7f6128f47700 1 -- 172.21.15.89:0/654519412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c009b30 con 0x7f6124070a40 2023-07-06T02:22:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.543+0000 7f6128f47700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f612415f0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f611800c450 tx=0x7f611800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.543+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180054e0 con 0x7f6124070a40 2023-07-06T02:22:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.543+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118003680 con 0x7f6124070a40 2023-07-06T02:22:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.543+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118003a30 con 0x7f6124070a40 2023-07-06T02:22:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.543+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612415e5e0 con 0x7f6124070a40 2023-07-06T02:22:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.543+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61240cadb0 con 0x7f6124070a40 2023-07-06T02:22:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.544+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612415e440 con 0x7f6124070a40 2023-07-06T02:22:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.544+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f6118003b90 con 0x7f6124070a40 2023-07-06T02:22:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.544+0000 7f61217fa700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f610806ac80 0x7f610806d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.544+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6118088520 con 0x7f6124070a40 2023-07-06T02:22:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.544+0000 7f6123fff700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f610806ac80 0x7f610806d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.545+0000 7f6123fff700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f610806ac80 0x7f610806d130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61241567c0 tx=0x7f6110009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.549+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f612415e440 con 0x7f6124070a40 2023-07-06T02:22:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.754+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6118053dd0 con 0x7f6124070a40 2023-07-06T02:22:48.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.783+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f612415e440 con 0x7f6124070a40 2023-07-06T02:22:48.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.784+0000 7f61217fa700 1 -- 172.21.15.89:0/654519412 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f6118017de0 con 0x7f6124070a40 2023-07-06T02:22:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:48.788 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-06T02:22:48.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f610806ac80 msgr2=0x7f610806d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f610806ac80 0x7f610806d130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61241567c0 tx=0x7f6110009040 comp rx=0 tx=0).stop 2023-07-06T02:22:48.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 msgr2=0x7f612415f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f612415f0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f611800c450 tx=0x7f611800c810 comp rx=0 tx=0).stop 2023-07-06T02:22:48.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 shutdown_connections 2023-07-06T02:22:48.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f612415c610 0x7f61240c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f610806ac80 0x7f610806d130 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6124070a40 0x7f612415f0c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 --2- 172.21.15.89:0/654519412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61240b5e50 0x7f61240ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 >> 172.21.15.89:0/654519412 conn(0x7f61240d0b00 msgr2=0x7f6124156810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:48.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 shutdown_connections 2023-07-06T02:22:48.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.786+0000 7f612b1ab700 1 -- 172.21.15.89:0/654519412 wait complete. 2023-07-06T02:22:48.793 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 3 2023-07-06T02:22:48.801 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:22:48.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.917+0000 7fa94e934700 1 Processor -- start 2023-07-06T02:22:48.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.918+0000 7fa94e934700 1 -- start start 2023-07-06T02:22:48.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.918+0000 7fa94e934700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa9480b1490 0x7fa9480b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.918+0000 7fa94e934700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa94806bfa0 0x7fa94806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.918+0000 7fa94e934700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9480cab50 con 0x7fa94815c690 2023-07-06T02:22:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.918+0000 7fa94e934700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94815de50 con 0x7fa94806bfa0 2023-07-06T02:22:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.918+0000 7fa94e934700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9480b1e50 con 0x7fa9480b1490 2023-07-06T02:22:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.918+0000 7fa94ced1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa94806bfa0 0x7fa94806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa94ced1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa94806bfa0 0x7fa94806c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60064/0 (socket says 172.21.15.89:60064) 2023-07-06T02:22:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa94ced1700 1 -- 172.21.15.89:0/3342850096 learned_addr learned my addr 172.21.15.89:0/3342850096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa9477fe700 1 --2- 172.21.15.89:0/3342850096 >> v2:172.21.15.172:3300/0 conn(0x7fa9480b1490 0x7fa9480b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa94ced1700 1 -- 172.21.15.89:0/3342850096 >> v2:172.21.15.172:3300/0 conn(0x7fa9480b1490 msgr2=0x7fa9480b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa94ced1700 1 --2- 172.21.15.89:0/3342850096 >> v2:172.21.15.172:3300/0 conn(0x7fa9480b1490 0x7fa9480b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa94ced1700 1 -- 172.21.15.89:0/3342850096 >> v1:172.21.15.172:6789/0 conn(0x7fa94815c690 legacy=0x7fa9480cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa94ced1700 1 -- 172.21.15.89:0/3342850096 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94806c8e0 con 0x7fa94806bfa0 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa94ced1700 1 --2- 172.21.15.89:0/3342850096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa94806c3a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa948070180 tx=0x7fa94000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646818c14ac39a06 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa946ffd700 1 -- 172.21.15.89:0/3342850096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940010070 con 0x7fa94806bfa0 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa946ffd700 1 -- 172.21.15.89:0/3342850096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000dd50 con 0x7fa94806bfa0 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.919+0000 7fa946ffd700 1 -- 172.21.15.89:0/3342850096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94000b310 con 0x7fa94806bfa0 2023-07-06T02:22:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 -- 172.21.15.89:0/3342850096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 msgr2=0x7fa94806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 --2- 172.21.15.89:0/3342850096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa94806c3a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa948070180 tx=0x7fa94000d430 comp rx=0 tx=0).stop 2023-07-06T02:22:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 -- 172.21.15.89:0/3342850096 shutdown_connections 2023-07-06T02:22:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 --2- 172.21.15.89:0/3342850096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa94806c3a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 --2- 172.21.15.89:0/3342850096 >> v2:172.21.15.172:3300/0 conn(0x7fa9480b1490 0x7fa9480b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 -- 172.21.15.89:0/3342850096 >> 172.21.15.89:0/3342850096 conn(0x7fa9480cc490 msgr2=0x7fa9480cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 -- 172.21.15.89:0/3342850096 shutdown_connections 2023-07-06T02:22:48.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.920+0000 7fa94e934700 1 -- 172.21.15.89:0/3342850096 wait complete. 2023-07-06T02:22:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.921+0000 7fa94e934700 1 Processor -- start 2023-07-06T02:22:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.921+0000 7fa94e934700 1 -- start start 2023-07-06T02:22:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa94e934700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa948152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa94e934700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9480b1490 0x7fa9480c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa94e934700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa94815c690 0x7fa9480b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa94e934700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9481531f0 con 0x7fa94806bfa0 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa94e934700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480b2cf0 con 0x7fa9480b1490 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa94e934700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480b2fd0 con 0x7fa94815c690 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa947fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa948152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa947fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa948152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60066/0 (socket says 172.21.15.89:60066) 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.922+0000 7fa947fff700 1 -- 172.21.15.89:0/2427994329 learned_addr learned my addr 172.21.15.89:0/2427994329 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa9477fe700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9480b1490 0x7fa9480c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa94ced1700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa94815c690 0x7fa9480b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa947fff700 1 -- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa94815c690 msgr2=0x7fa9480b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa947fff700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa94815c690 0x7fa9480b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa947fff700 1 -- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9480b1490 msgr2=0x7fa9480c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa947fff700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9480b1490 0x7fa9480c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa947fff700 1 -- 172.21.15.89:0/2427994329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94000bd40 con 0x7fa94806bfa0 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa9477fe700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9480b1490 0x7fa9480c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa947fff700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa948152cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c4d0 tx=0x7fa93800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa944ff9700 1 -- 172.21.15.89:0/2427994329 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938011790 con 0x7fa94806bfa0 2023-07-06T02:22:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa944ff9700 1 -- 172.21.15.89:0/2427994329 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9380118f0 con 0x7fa94806bfa0 2023-07-06T02:22:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9480b32b0 con 0x7fa94806bfa0 2023-07-06T02:22:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa944ff9700 1 -- 172.21.15.89:0/2427994329 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938011ca0 con 0x7fa94806bfa0 2023-07-06T02:22:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.923+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9480b3870 con 0x7fa94806bfa0 2023-07-06T02:22:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.924+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa948074270 con 0x7fa94806bfa0 2023-07-06T02:22:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.924+0000 7fa944ff9700 1 -- 172.21.15.89:0/2427994329 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa93801c460 con 0x7fa94806bfa0 2023-07-06T02:22:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.925+0000 7fa944ff9700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa92806ace0 0x7fa92806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.925+0000 7fa9477fe700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa92806ace0 0x7fa92806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.925+0000 7fa944ff9700 1 -- 172.21.15.89:0/2427994329 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa938088f70 con 0x7fa94806bfa0 2023-07-06T02:22:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.925+0000 7fa9477fe700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa92806ace0 0x7fa92806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa94806bcb0 tx=0x7fa930008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:48.929+0000 7fa944ff9700 1 -- 172.21.15.89:0/2427994329 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9380bb310 con 0x7fa94806bfa0 2023-07-06T02:22:49.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.116+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9480c32a0 con 0x7fa94806bfa0 2023-07-06T02:22:49.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.117+0000 7fa944ff9700 1 -- 172.21.15.89:0/2427994329 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14567 (secure 0 0 0) 0x7fa938054370 con 0x7fa94806bfa0 2023-07-06T02:22:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:49.121 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":16,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:48.374373+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:22:49.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa92806ace0 msgr2=0x7fa92806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa92806ace0 0x7fa92806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa94806bcb0 tx=0x7fa930008040 comp rx=0 tx=0).stop 2023-07-06T02:22:49.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 msgr2=0x7fa948152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa948152cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c4d0 tx=0x7fa93800c890 comp rx=0 tx=0).stop 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 shutdown_connections 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa94815c690 0x7fa9480b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa92806ace0 0x7fa92806d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa94806bfa0 0x7fa948152cb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 --2- 172.21.15.89:0/2427994329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9480b1490 0x7fa9480c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 >> 172.21.15.89:0/2427994329 conn(0x7fa9480cc490 msgr2=0x7fa9480d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:49.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 shutdown_connections 2023-07-06T02:22:49.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.119+0000 7fa94e934700 1 -- 172.21.15.89:0/2427994329 wait complete. 2023-07-06T02:22:49.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T02:22:49.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:22:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.256+0000 7fd30cae4700 1 Processor -- start 2023-07-06T02:22:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.256+0000 7fd30cae4700 1 -- start start 2023-07-06T02:22:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30cae4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3081600b0 0x7fd3080b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30cae4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd3080b19d0 0x7fd3080cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30cae4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3080cb3d0 con 0x7fd308156ab0 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30cae4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3080b1df0 con 0x7fd3080b19d0 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30cae4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd308156e80 con 0x7fd3081600b0 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd308156ab0 0x7fd308160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56744/0 (socket says 172.21.15.89:56744) 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30659c700 1 -- 172.21.15.89:0/1197051720 learned_addr learned my addr 172.21.15.89:0/1197051720 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd306d9d700 1 --2- 172.21.15.89:0/1197051720 >> v2:172.21.15.172:3300/0 conn(0x7fd3080b19d0 0x7fd3080cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd305d9b700 1 --2- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3081600b0 0x7fd3080b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30559a700 1 -- 172.21.15.89:0/1197051720 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836084481 0 0) 0x7fd3080cb3d0 con 0x7fd308156ab0 2023-07-06T02:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30559a700 1 -- 172.21.15.89:0/1197051720 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f00035b0 con 0x7fd308156ab0 2023-07-06T02:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd30559a700 1 -- 172.21.15.89:0/1197051720 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1552659828 0 0) 0x7fd2f00035b0 con 0x7fd308156ab0 2023-07-06T02:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd306d9d700 1 -- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3081600b0 msgr2=0x7fd3080b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd306d9d700 1 --2- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3081600b0 0x7fd3080b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd306d9d700 1 -- 172.21.15.89:0/1197051720 >> v1:172.21.15.89:6789/0 conn(0x7fd308156ab0 legacy=0x7fd308160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd306d9d700 1 -- 172.21.15.89:0/1197051720 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3080ca370 con 0x7fd3080b19d0 2023-07-06T02:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.257+0000 7fd305d9b700 1 --2- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3081600b0 0x7fd3080b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd306d9d700 1 --2- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3080b19d0 0x7fd3080cd700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd300009d90 tx=0x7fd30000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e8e4b210594f1b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30559a700 1 -- 172.21.15.89:0/1197051720 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000e070 con 0x7fd3080b19d0 2023-07-06T02:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30559a700 1 -- 172.21.15.89:0/1197051720 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300003a20 con 0x7fd3080b19d0 2023-07-06T02:22:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30559a700 1 -- 172.21.15.89:0/1197051720 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8001670 con 0x7fd3080b19d0 2023-07-06T02:22:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 -- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3080b19d0 msgr2=0x7fd3080cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 --2- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3080b19d0 0x7fd3080cd700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd300009d90 tx=0x7fd30000b330 comp rx=0 tx=0).stop 2023-07-06T02:22:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 -- 172.21.15.89:0/1197051720 shutdown_connections 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 --2- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3080b19d0 0x7fd3080cd700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 --2- 172.21.15.89:0/1197051720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3081600b0 0x7fd3080b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 -- 172.21.15.89:0/1197051720 >> 172.21.15.89:0/1197051720 conn(0x7fd3080cc490 msgr2=0x7fd3080cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 -- 172.21.15.89:0/1197051720 shutdown_connections 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.258+0000 7fd30cae4700 1 -- 172.21.15.89:0/1197051720 wait complete. 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 Processor -- start 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 -- start start 2023-07-06T02:22:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3080b19d0 0x7fd3080c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308156ab0 0x7fd3081e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3081600b0 0x7fd30815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080b3270 con 0x7fd308156ab0 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080b3550 con 0x7fd3081600b0 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.260+0000 7fd30cae4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080b3830 con 0x7fd3080b19d0 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308156ab0 0x7fd3081e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308156ab0 0x7fd3081e5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60082/0 (socket says 172.21.15.89:60082) 2023-07-06T02:22:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 -- 172.21.15.89:0/661216094 learned_addr learned my addr 172.21.15.89:0/661216094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd306d9d700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3081600b0 0x7fd30815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd30659c700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3080b19d0 0x7fd3080c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 -- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3080b19d0 msgr2=0x7fd3080c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3080b19d0 0x7fd3080c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 -- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3081600b0 msgr2=0x7fd30815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3081600b0 0x7fd30815cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 -- 172.21.15.89:0/661216094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0094a0 con 0x7fd308156ab0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd305d9b700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308156ab0 0x7fd3081e5dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00ebe0 tx=0x7fd2fc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd2f6ffd700 1 -- 172.21.15.89:0/661216094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00c850 con 0x7fd308156ab0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd2f6ffd700 1 -- 172.21.15.89:0/661216094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00c9b0 con 0x7fd308156ab0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd300009a70 con 0x7fd308156ab0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd2f6ffd700 1 -- 172.21.15.89:0/661216094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00cd40 con 0x7fd308156ab0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.261+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3080b3df0 con 0x7fd308156ab0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.262+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308074270 con 0x7fd308156ab0 2023-07-06T02:22:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.262+0000 7fd2f6ffd700 1 -- 172.21.15.89:0/661216094 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2fc013470 con 0x7fd308156ab0 2023-07-06T02:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.263+0000 7fd2f6ffd700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.263+0000 7fd2f6ffd700 1 -- 172.21.15.89:0/661216094 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd2fc0891e0 con 0x7fd308156ab0 2023-07-06T02:22:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.263+0000 7fd30659c700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.263+0000 7fd30659c700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e806ace0 0x7fd2e806d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8001eb0 tx=0x7fd2f8001e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.267+0000 7fd2f6ffd700 1 -- 172.21.15.89:0/661216094 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd2fc0545e0 con 0x7fd308156ab0 2023-07-06T02:22:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.457+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3080b5990 con 0x7fd308156ab0 2023-07-06T02:22:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.458+0000 7fd2f6ffd700 1 -- 172.21.15.89:0/661216094 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2fc00cea0 con 0x7fd308156ab0 2023-07-06T02:22:49.461 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:22:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e806ace0 msgr2=0x7fd2e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e806ace0 0x7fd2e806d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8001eb0 tx=0x7fd2f8001e40 comp rx=0 tx=0).stop 2023-07-06T02:22:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308156ab0 msgr2=0x7fd3081e5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308156ab0 0x7fd3081e5dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00ebe0 tx=0x7fd2fc00eef0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 shutdown_connections 2023-07-06T02:22:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3081600b0 0x7fd30815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3080b19d0 0x7fd3080c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 --2- 172.21.15.89:0/661216094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd308156ab0 0x7fd3081e5dd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 >> 172.21.15.89:0/661216094 conn(0x7fd3080cc490 msgr2=0x7fd30815f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 shutdown_connections 2023-07-06T02:22:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.460+0000 7fd30cae4700 1 -- 172.21.15.89:0/661216094 wait complete. 2023-07-06T02:22:49.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:22:49.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.596+0000 7fe36272e700 1 Processor -- start 2023-07-06T02:22:49.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.597+0000 7fe36272e700 1 -- start start 2023-07-06T02:22:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.597+0000 7fe36272e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe35c156ab0 0x7fe35c160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.597+0000 7fe36272e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe35c160bb0 0x7fe35c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.597+0000 7fe36272e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.597+0000 7fe36272e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c0cb3d0 con 0x7fe35c160bb0 2023-07-06T02:22:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.597+0000 7fe36272e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c156e80 con 0x7fe35c156ab0 2023-07-06T02:22:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.597+0000 7fe36272e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0ca370 con 0x7fe35c1600b0 2023-07-06T02:22:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe35c160bb0 0x7fe35c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe35c160bb0 0x7fe35c0b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60088/0 (socket says 172.21.15.89:60088) 2023-07-06T02:22:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 -- 172.21.15.89:0/1629257310 learned_addr learned my addr 172.21.15.89:0/1629257310 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35bfff700 1 --2- 172.21.15.89:0/1629257310 >> v2:172.21.15.172:3300/0 conn(0x7fe35c156ab0 0x7fe35c160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe360ccb700 1 --2- 172.21.15.89:0/1629257310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 -- 172.21.15.89:0/1629257310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 msgr2=0x7fe35c0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 --2- 172.21.15.89:0/1629257310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c0b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 -- 172.21.15.89:0/1629257310 >> v2:172.21.15.172:3300/0 conn(0x7fe35c156ab0 msgr2=0x7fe35c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 --2- 172.21.15.89:0/1629257310 >> v2:172.21.15.172:3300/0 conn(0x7fe35c156ab0 0x7fe35c160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 -- 172.21.15.89:0/1629257310 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0ca610 con 0x7fe35c160bb0 2023-07-06T02:22:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35bfff700 1 --2- 172.21.15.89:0/1629257310 >> v2:172.21.15.172:3300/0 conn(0x7fe35c156ab0 0x7fe35c160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35b7fe700 1 --2- 172.21.15.89:0/1629257310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c160bb0 0x7fe35c0b1490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe35c06e9e0 tx=0x7fe34c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ac5be6c4dd8ec8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35affd700 1 -- 172.21.15.89:0/1629257310 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00b870 con 0x7fe35c160bb0 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35affd700 1 -- 172.21.15.89:0/1629257310 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c00b9d0 con 0x7fe35c160bb0 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.598+0000 7fe35affd700 1 -- 172.21.15.89:0/1629257310 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00bd60 con 0x7fe35c160bb0 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 -- 172.21.15.89:0/1629257310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c160bb0 msgr2=0x7fe35c0b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 --2- 172.21.15.89:0/1629257310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c160bb0 0x7fe35c0b1490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe35c06e9e0 tx=0x7fe34c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 -- 172.21.15.89:0/1629257310 shutdown_connections 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 --2- 172.21.15.89:0/1629257310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 --2- 172.21.15.89:0/1629257310 >> v2:172.21.15.172:3300/0 conn(0x7fe35c156ab0 0x7fe35c160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 --2- 172.21.15.89:0/1629257310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c160bb0 0x7fe35c0b1490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 -- 172.21.15.89:0/1629257310 >> 172.21.15.89:0/1629257310 conn(0x7fe35c0cc490 msgr2=0x7fe35c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 -- 172.21.15.89:0/1629257310 shutdown_connections 2023-07-06T02:22:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.599+0000 7fe36272e700 1 -- 172.21.15.89:0/1629257310 wait complete. 2023-07-06T02:22:49.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.600+0000 7fe36272e700 1 Processor -- start 2023-07-06T02:22:49.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.600+0000 7fe36272e700 1 -- start start 2023-07-06T02:22:49.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c156ab0 0x7fe35c0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c1f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c155620 0x7fe35c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c85b0 con 0x7fe35c155620 2023-07-06T02:22:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1f9680 con 0x7fe35c156ab0 2023-07-06T02:22:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1fa690 con 0x7fe35c1600b0 2023-07-06T02:22:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c155620 0x7fe35c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c155620 0x7fe35c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60096/0 (socket says 172.21.15.89:60096) 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 -- 172.21.15.89:0/322020515 learned_addr learned my addr 172.21.15.89:0/322020515 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe35bfff700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c156ab0 0x7fe35c0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe35b7fe700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c1f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 -- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 msgr2=0x7fe35c1f9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c1f9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 -- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c156ab0 msgr2=0x7fe35c0b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c156ab0 0x7fe35c0b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 -- 172.21.15.89:0/322020515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c009550 con 0x7fe35c155620 2023-07-06T02:22:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe35bfff700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c156ab0 0x7fe35c0b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe360ccb700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c155620 0x7fe35c155aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe35400cd40 tx=0x7fe35400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe358ff9700 1 -- 172.21.15.89:0/322020515 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354003a20 con 0x7fe35c155620 2023-07-06T02:22:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe358ff9700 1 -- 172.21.15.89:0/322020515 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354003b80 con 0x7fe35c155620 2023-07-06T02:22:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0c7930 con 0x7fe35c155620 2023-07-06T02:22:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe358ff9700 1 -- 172.21.15.89:0/322020515 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354004020 con 0x7fe35c155620 2023-07-06T02:22:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.601+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0c81d0 con 0x7fe35c155620 2023-07-06T02:22:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.602+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c1f74e0 con 0x7fe35c155620 2023-07-06T02:22:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.603+0000 7fe358ff9700 1 -- 172.21.15.89:0/322020515 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe354004180 con 0x7fe35c155620 2023-07-06T02:22:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.603+0000 7fe358ff9700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe34806ace0 0x7fe34806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.603+0000 7fe358ff9700 1 -- 172.21.15.89:0/322020515 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe354012070 con 0x7fe35c155620 2023-07-06T02:22:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.603+0000 7fe35bfff700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe34806ace0 0x7fe34806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.604+0000 7fe35bfff700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe34806ace0 0x7fe34806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3500089b0 tx=0x7fe350008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.608+0000 7fe358ff9700 1 -- 172.21.15.89:0/322020515 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe354054db0 con 0x7fe35c155620 2023-07-06T02:22:49.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.793+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe35c0c7d30 con 0x7fe35c155620 2023-07-06T02:22:49.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.793+0000 7fe358ff9700 1 -- 172.21.15.89:0/322020515 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3540583d0 con 0x7fe35c155620 2023-07-06T02:22:49.796 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:22:49.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe34806ace0 msgr2=0x7fe34806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe34806ace0 0x7fe34806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3500089b0 tx=0x7fe350008040 comp rx=0 tx=0).stop 2023-07-06T02:22:49.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c155620 msgr2=0x7fe35c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c155620 0x7fe35c155aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe35400cd40 tx=0x7fe35400a570 comp rx=0 tx=0).stop 2023-07-06T02:22:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 shutdown_connections 2023-07-06T02:22:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe34806ace0 0x7fe34806d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c155620 0x7fe35c155aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe35c156ab0 0x7fe35c0b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 --2- 172.21.15.89:0/322020515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe35c1600b0 0x7fe35c1f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 >> 172.21.15.89:0/322020515 conn(0x7fe35c0cc490 msgr2=0x7fe35c0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 shutdown_connections 2023-07-06T02:22:49.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.795+0000 7fe36272e700 1 -- 172.21.15.89:0/322020515 wait complete. 2023-07-06T02:22:49.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T02:22:49.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.927+0000 7f2a67cfa700 1 Processor -- start 2023-07-06T02:22:49.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.927+0000 7f2a67cfa700 1 -- start start 2023-07-06T02:22:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a67cfa700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a600b1490 0x7f2a600caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a67cfa700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a6006b220 0x7f2a6015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a67cfa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6006a990 con 0x7f2a600cc000 2023-07-06T02:22:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a67cfa700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6006ac70 con 0x7f2a600b1490 2023-07-06T02:22:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a67cfa700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6006d170 con 0x7f2a6006b220 2023-07-06T02:22:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65a96700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a600b1490 0x7f2a600caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65295700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2a600cc000 0x7f2a600b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56774/0 (socket says 172.21.15.89:56774) 2023-07-06T02:22:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65a96700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a600b1490 0x7f2a600caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60100/0 (socket says 172.21.15.89:60100) 2023-07-06T02:22:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65295700 1 -- 172.21.15.89:0/218192715 learned_addr learned my addr 172.21.15.89:0/218192715 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:49.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a66297700 1 --2- 172.21.15.89:0/218192715 >> v2:172.21.15.172:3300/0 conn(0x7f2a6006b220 0x7f2a6015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65a96700 1 -- 172.21.15.89:0/218192715 >> v2:172.21.15.172:3300/0 conn(0x7f2a6006b220 msgr2=0x7f2a6015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65a96700 1 --2- 172.21.15.89:0/218192715 >> v2:172.21.15.172:3300/0 conn(0x7f2a6006b220 0x7f2a6015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65a96700 1 -- 172.21.15.89:0/218192715 >> v1:172.21.15.89:6789/0 conn(0x7f2a600cc000 legacy=0x7f2a600b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65a96700 1 -- 172.21.15.89:0/218192715 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6006ce60 con 0x7f2a600b1490 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a64a94700 1 -- 172.21.15.89:0/218192715 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593529451 0 0) 0x7f2a6006a990 con 0x7f2a600cc000 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a65a96700 1 --2- 172.21.15.89:0/218192715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a600b1490 0x7f2a600caeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b2460 tx=0x7f2a50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02cde5a62c45e28 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a64a94700 1 -- 172.21.15.89:0/218192715 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011830 con 0x7f2a600b1490 2023-07-06T02:22:49.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a64a94700 1 -- 172.21.15.89:0/218192715 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50011990 con 0x7f2a600b1490 2023-07-06T02:22:49.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.928+0000 7f2a64a94700 1 -- 172.21.15.89:0/218192715 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011d20 con 0x7f2a600b1490 2023-07-06T02:22:49.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 -- 172.21.15.89:0/218192715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a600b1490 msgr2=0x7f2a600caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/218192715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a600b1490 0x7f2a600caeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b2460 tx=0x7f2a50009b20 comp rx=0 tx=0).stop 2023-07-06T02:22:49.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 -- 172.21.15.89:0/218192715 shutdown_connections 2023-07-06T02:22:49.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/218192715 >> v2:172.21.15.172:3300/0 conn(0x7f2a6006b220 0x7f2a6015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/218192715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a600b1490 0x7f2a600caeb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 -- 172.21.15.89:0/218192715 >> 172.21.15.89:0/218192715 conn(0x7f2a600cc490 msgr2=0x7f2a600cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:49.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 -- 172.21.15.89:0/218192715 shutdown_connections 2023-07-06T02:22:49.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.929+0000 7f2a67cfa700 1 -- 172.21.15.89:0/218192715 wait complete. 2023-07-06T02:22:49.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.930+0000 7f2a67cfa700 1 Processor -- start 2023-07-06T02:22:49.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.930+0000 7f2a67cfa700 1 -- start start 2023-07-06T02:22:49.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a6006b220 0x7f2a600ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b1490 0x7f2a60150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600cc000 0x7f2a600b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600aeca0 con 0x7f2a6006b220 2023-07-06T02:22:49.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60150580 con 0x7f2a600b1490 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6015ceb0 con 0x7f2a600cc000 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a6006b220 0x7f2a600ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a6006b220 0x7f2a600ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60106/0 (socket says 172.21.15.89:60106) 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 -- 172.21.15.89:0/4166456400 learned_addr learned my addr 172.21.15.89:0/4166456400 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 -- 172.21.15.89:0/4166456400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600cc000 msgr2=0x7f2a600b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600cc000 0x7f2a600b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 -- 172.21.15.89:0/4166456400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b1490 msgr2=0x7f2a60150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:49.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b1490 0x7f2a60150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 -- 172.21.15.89:0/4166456400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500094a0 con 0x7f2a6006b220 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a65a96700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a6006b220 0x7f2a600ae760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a500090b0 tx=0x7f2a50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a567fc700 1 -- 172.21.15.89:0/4166456400 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500038a0 con 0x7f2a6006b220 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a567fc700 1 -- 172.21.15.89:0/4166456400 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5001f070 con 0x7f2a6006b220 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a567fc700 1 -- 172.21.15.89:0/4166456400 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50003c00 con 0x7f2a6006b220 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6015c8f0 con 0x7f2a6006b220 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.931+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600c7600 con 0x7f2a6006b220 2023-07-06T02:22:49.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.932+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a600b5400 con 0x7f2a6006b220 2023-07-06T02:22:49.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.933+0000 7f2a567fc700 1 -- 172.21.15.89:0/4166456400 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a5001a430 con 0x7f2a6006b220 2023-07-06T02:22:49.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.933+0000 7f2a567fc700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4406ace0 0x7f2a4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:49.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.933+0000 7f2a65295700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4406ace0 0x7f2a4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:49.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.934+0000 7f2a567fc700 1 -- 172.21.15.89:0/4166456400 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2a50021070 con 0x7f2a6006b220 2023-07-06T02:22:49.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.934+0000 7f2a65295700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4406ace0 0x7f2a4406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0011e0 tx=0x7f2a5c0079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:49.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:49.938+0000 7f2a567fc700 1 -- 172.21.15.89:0/4166456400 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a50055fd0 con 0x7f2a6006b220 2023-07-06T02:22:50.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.131+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2a60074270 con 0x7f2a6006b220 2023-07-06T02:22:50.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.132+0000 7f2a567fc700 1 -- 172.21.15.89:0/4166456400 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2a5002b090 con 0x7f2a6006b220 2023-07-06T02:22:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:50.135 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-07-06T02:22:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:22:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-07-06T02:22:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": "rbd" 2023-07-06T02:22:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:22:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:22:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 2, 2023-07-06T02:22:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-07-06T02:22:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:22:50.136 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-07-06T02:22:50.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4406ace0 msgr2=0x7f2a4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:50.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4406ace0 0x7f2a4406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0011e0 tx=0x7f2a5c0079b0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a6006b220 msgr2=0x7f2a600ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:50.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a6006b220 0x7f2a600ae760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a500090b0 tx=0x7f2a50004750 comp rx=0 tx=0).stop 2023-07-06T02:22:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 shutdown_connections 2023-07-06T02:22:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600cc000 0x7f2a600b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4406ace0 0x7f2a4406d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a6006b220 0x7f2a600ae760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 --2- 172.21.15.89:0/4166456400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b1490 0x7f2a60150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 >> 172.21.15.89:0/4166456400 conn(0x7f2a600cc490 msgr2=0x7f2a6006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 shutdown_connections 2023-07-06T02:22:50.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.134+0000 7f2a67cfa700 1 -- 172.21.15.89:0/4166456400 wait complete. 2023-07-06T02:22:50.148 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T02:22:50.149 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T02:22:50.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.269+0000 7fa7977a6700 1 Processor -- start 2023-07-06T02:22:50.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.269+0000 7fa7977a6700 1 -- start start 2023-07-06T02:22:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa7977a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa790071670 0x7fa790071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa7977a6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa790072030 con 0x7fa790156c30 2023-07-06T02:22:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa7977a6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa790157000 con 0x7fa7900ca0c0 2023-07-06T02:22:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa7977a6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7900cdd00 con 0x7fa790071670 2023-07-06T02:22:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa794d41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa790071670 0x7fa790071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa795542700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa790156c30 0x7fa7900cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56784/0 (socket says 172.21.15.89:56784) 2023-07-06T02:22:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa794d41700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa790071670 0x7fa790071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60114/0 (socket says 172.21.15.89:60114) 2023-07-06T02:22:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.270+0000 7fa795542700 1 -- 172.21.15.89:0/2354516632 learned_addr learned my addr 172.21.15.89:0/2354516632 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa794d41700 1 -- 172.21.15.89:0/2354516632 >> v1:172.21.15.172:6789/0 conn(0x7fa7900ca0c0 legacy=0x7fa7900ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa794d41700 1 -- 172.21.15.89:0/2354516632 >> v1:172.21.15.89:6789/0 conn(0x7fa790156c30 legacy=0x7fa7900cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa787fff700 1 -- 172.21.15.89:0/2354516632 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458361936 0 0) 0x7fa790072030 con 0x7fa790156c30 2023-07-06T02:22:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa794d41700 1 -- 172.21.15.89:0/2354516632 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7901602c0 con 0x7fa790071670 2023-07-06T02:22:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa794d41700 1 --2- 172.21.15.89:0/2354516632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa790071670 0x7fa790071af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009840 tx=0x7fa78c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb4ba1ddb7b87d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa787fff700 1 -- 172.21.15.89:0/2354516632 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c004230 con 0x7fa790071670 2023-07-06T02:22:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa787fff700 1 -- 172.21.15.89:0/2354516632 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c004390 con 0x7fa790071670 2023-07-06T02:22:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.271+0000 7fa787fff700 1 -- 172.21.15.89:0/2354516632 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0036a0 con 0x7fa790071670 2023-07-06T02:22:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.272+0000 7fa7977a6700 1 -- 172.21.15.89:0/2354516632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa790071670 msgr2=0x7fa790071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.272+0000 7fa7977a6700 1 --2- 172.21.15.89:0/2354516632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa790071670 0x7fa790071af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009840 tx=0x7fa78c009b50 comp rx=0 tx=0).stop 2023-07-06T02:22:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.272+0000 7fa7977a6700 1 -- 172.21.15.89:0/2354516632 shutdown_connections 2023-07-06T02:22:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.272+0000 7fa7977a6700 1 --2- 172.21.15.89:0/2354516632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa790071670 0x7fa790071af0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.272+0000 7fa7977a6700 1 -- 172.21.15.89:0/2354516632 >> 172.21.15.89:0/2354516632 conn(0x7fa7900cf970 msgr2=0x7fa7900cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.272+0000 7fa7977a6700 1 -- 172.21.15.89:0/2354516632 shutdown_connections 2023-07-06T02:22:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.272+0000 7fa7977a6700 1 -- 172.21.15.89:0/2354516632 wait complete. 2023-07-06T02:22:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.273+0000 7fa7977a6700 1 Processor -- start 2023-07-06T02:22:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.273+0000 7fa7977a6700 1 -- start start 2023-07-06T02:22:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa790071670 0x7fa79013f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7900ca0c0 0x7fa7901f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa790156c30 0x7fa7901556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901f6a40 con 0x7fa7900ca0c0 2023-07-06T02:22:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901f6480 con 0x7fa790071670 2023-07-06T02:22:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901f6760 con 0x7fa790156c30 2023-07-06T02:22:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7900ca0c0 0x7fa7901f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:50.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7900ca0c0 0x7fa7901f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60122/0 (socket says 172.21.15.89:60122) 2023-07-06T02:22:50.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 -- 172.21.15.89:0/148409410 learned_addr learned my addr 172.21.15.89:0/148409410 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:50.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa795542700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa790071670 0x7fa79013f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:50.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 -- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa790156c30 msgr2=0x7fa7901556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa790156c30 0x7fa7901556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 -- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa790071670 msgr2=0x7fa79013f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa790071670 0x7fa79013f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 -- 172.21.15.89:0/148409410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0094a0 con 0x7fa7900ca0c0 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa795542700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa790071670 0x7fa79013f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa794d41700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7900ca0c0 0x7fa7901f5aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0090b0 tx=0x7fa78c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa785ffb700 1 -- 172.21.15.89:0/148409410 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c011830 con 0x7fa7900ca0c0 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa785ffb700 1 -- 172.21.15.89:0/148409410 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c01e070 con 0x7fa7900ca0c0 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7900c6be0 con 0x7fa7900ca0c0 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa785ffb700 1 -- 172.21.15.89:0/148409410 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c011bf0 con 0x7fa7900ca0c0 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.274+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79013ff00 con 0x7fa7900ca0c0 2023-07-06T02:22:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.275+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7900c7100 con 0x7fa7900ca0c0 2023-07-06T02:22:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.276+0000 7fa785ffb700 1 -- 172.21.15.89:0/148409410 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa78c011d50 con 0x7fa7900ca0c0 2023-07-06T02:22:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.276+0000 7fa785ffb700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa77406ace0 0x7fa77406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.276+0000 7fa785ffb700 1 -- 172.21.15.89:0/148409410 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa78c08a460 con 0x7fa7900ca0c0 2023-07-06T02:22:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.276+0000 7fa795542700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa77406ace0 0x7fa77406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.276+0000 7fa795542700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa77406ace0 0x7fa77406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa780007fd0 tx=0x7fa780009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.280+0000 7fa785ffb700 1 -- 172.21.15.89:0/148409410 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7900c7100 con 0x7fa7900ca0c0 2023-07-06T02:22:50.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:50.469+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa7900c7100 con 0x7fa7900ca0c0 2023-07-06T02:22:51.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.396+0000 7fa785ffb700 1 -- 172.21.15.89:0/148409410 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa7900c7100 con 0x7fa7900ca0c0 2023-07-06T02:22:51.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa77406ace0 msgr2=0x7fa77406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:51.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa77406ace0 0x7fa77406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa780007fd0 tx=0x7fa780009040 comp rx=0 tx=0).stop 2023-07-06T02:22:51.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7900ca0c0 msgr2=0x7fa7901f5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:51.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7900ca0c0 0x7fa7901f5aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0090b0 tx=0x7fa78c004750 comp rx=0 tx=0).stop 2023-07-06T02:22:51.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 shutdown_connections 2023-07-06T02:22:51.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa790156c30 0x7fa7901556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa790071670 0x7fa79013f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa77406ace0 0x7fa77406d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 --2- 172.21.15.89:0/148409410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7900ca0c0 0x7fa7901f5aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 >> 172.21.15.89:0/148409410 conn(0x7fa7900cf970 msgr2=0x7fa790071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:51.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 shutdown_connections 2023-07-06T02:22:51.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.400+0000 7fa7977a6700 1 -- 172.21.15.89:0/148409410 wait complete. 2023-07-06T02:22:51.408 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-07-06T02:22:51.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-06T02:22:51.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1256fc4700 1 Processor -- start 2023-07-06T02:22:51.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1256fc4700 1 -- start start 2023-07-06T02:22:51.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1256fc4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12501600b0 0x7f12500b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:51.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1256fc4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1256fc4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12500cb3d0 con 0x7f1250156ab0 2023-07-06T02:22:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1256fc4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12500b1e10 con 0x7f12501600b0 2023-07-06T02:22:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1256fc4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1250156e80 con 0x7f12500b19d0 2023-07-06T02:22:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1255561700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f124ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12501600b0 0x7f12500b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f124ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12501600b0 0x7f12500b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:58888/0 (socket says 172.21.15.89:58888) 2023-07-06T02:22:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1255561700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55108/0 (socket says 172.21.15.89:55108) 2023-07-06T02:22:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1254d60700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1250156ab0 0x7f1250160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:60786/0 (socket says 172.21.15.89:60786) 2023-07-06T02:22:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f124ffff700 1 -- 172.21.15.89:0/3259258421 learned_addr learned my addr 172.21.15.89:0/3259258421 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1255561700 1 -- 172.21.15.89:0/3259258421 >> v2:172.21.15.172:3300/0 conn(0x7f12501600b0 msgr2=0x7f12500b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1255561700 1 --2- 172.21.15.89:0/3259258421 >> v2:172.21.15.172:3300/0 conn(0x7f12501600b0 0x7f12500b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1255561700 1 -- 172.21.15.89:0/3259258421 >> v1:172.21.15.172:6789/0 conn(0x7f1250156ab0 legacy=0x7f1250160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f1255561700 1 -- 172.21.15.89:0/3259258421 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500ca370 con 0x7f12500b19d0 2023-07-06T02:22:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.552+0000 7f124ffff700 1 --2- 172.21.15.89:0/3259258421 >> v2:172.21.15.172:3300/0 conn(0x7f12501600b0 0x7f12500b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1255561700 1 --2- 172.21.15.89:0/3259258421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500cd700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1248009d90 tx=0x7f124800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fbf68e88004a51b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f124f7fe700 1 -- 172.21.15.89:0/3259258421 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124800e070 con 0x7f12500b19d0 2023-07-06T02:22:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f124f7fe700 1 -- 172.21.15.89:0/3259258421 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124800bb20 con 0x7f12500b19d0 2023-07-06T02:22:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f124f7fe700 1 -- 172.21.15.89:0/3259258421 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248012430 con 0x7f12500b19d0 2023-07-06T02:22:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 -- 172.21.15.89:0/3259258421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 msgr2=0x7f12500cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 --2- 172.21.15.89:0/3259258421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500cd700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1248009d90 tx=0x7f124800b330 comp rx=0 tx=0).stop 2023-07-06T02:22:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 -- 172.21.15.89:0/3259258421 shutdown_connections 2023-07-06T02:22:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 --2- 172.21.15.89:0/3259258421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500cd700 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 --2- 172.21.15.89:0/3259258421 >> v2:172.21.15.172:3300/0 conn(0x7f12501600b0 0x7f12500b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 -- 172.21.15.89:0/3259258421 >> 172.21.15.89:0/3259258421 conn(0x7f12500cc490 msgr2=0x7f12500cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 -- 172.21.15.89:0/3259258421 shutdown_connections 2023-07-06T02:22:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.553+0000 7f1256fc4700 1 -- 172.21.15.89:0/3259258421 wait complete. 2023-07-06T02:22:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.554+0000 7f1256fc4700 1 Processor -- start 2023-07-06T02:22:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.555+0000 7f1256fc4700 1 -- start start 2023-07-06T02:22:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.555+0000 7f1256fc4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.555+0000 7f1256fc4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1250156ab0 0x7f1250154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.555+0000 7f1256fc4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12501600b0 0x7f12500b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.555+0000 7f1256fc4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500ad9b0 con 0x7f1250156ab0 2023-07-06T02:22:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.555+0000 7f1256fc4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1250155180 con 0x7f12501600b0 2023-07-06T02:22:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.555+0000 7f1256fc4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12500acda0 con 0x7f12500b19d0 2023-07-06T02:22:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1250156ab0 0x7f1250154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1250156ab0 0x7f1250154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60136/0 (socket says 172.21.15.89:60136) 2023-07-06T02:22:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 -- 172.21.15.89:0/2254351642 learned_addr learned my addr 172.21.15.89:0/2254351642 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f1255561700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12501600b0 0x7f12500b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f1254d60700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 -- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 msgr2=0x7f12500ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 -- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12501600b0 msgr2=0x7f12500b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12501600b0 0x7f12500b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 -- 172.21.15.89:0/2254351642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c009550 con 0x7f1250156ab0 2023-07-06T02:22:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124ffff700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1250156ab0 0x7f1250154c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f123c00e9b0 tx=0x7f123c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124d7fa700 1 -- 172.21.15.89:0/2254351642 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00ee50 con 0x7f1250156ab0 2023-07-06T02:22:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124d7fa700 1 -- 172.21.15.89:0/2254351642 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c00c850 con 0x7f1250156ab0 2023-07-06T02:22:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1248009a70 con 0x7f1250156ab0 2023-07-06T02:22:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f124d7fa700 1 -- 172.21.15.89:0/2254351642 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00cbc0 con 0x7f1250156ab0 2023-07-06T02:22:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.556+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12500ad360 con 0x7f1250156ab0 2023-07-06T02:22:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.557+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12500adb20 con 0x7f1250156ab0 2023-07-06T02:22:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.558+0000 7f124d7fa700 1 -- 172.21.15.89:0/2254351642 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f123c00cd20 con 0x7f1250156ab0 2023-07-06T02:22:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.558+0000 7f124d7fa700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f123406ace0 0x7f123406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.558+0000 7f1254d60700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f123406ace0 0x7f123406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.558+0000 7f124d7fa700 1 -- 172.21.15.89:0/2254351642 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f123c089260 con 0x7f1250156ab0 2023-07-06T02:22:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.559+0000 7f1254d60700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f123406ace0 0x7f123406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1244000f60 tx=0x7f1244007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.564+0000 7f124d7fa700 1 -- 172.21.15.89:0/2254351642 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f123c054580 con 0x7f1250156ab0 2023-07-06T02:22:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:51.751+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f1250074270 con 0x7f1250156ab0 2023-07-06T02:22:52.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.403+0000 7f124d7fa700 1 -- 172.21.15.89:0/2254351642 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f123c057ba0 con 0x7f1250156ab0 2023-07-06T02:22:52.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f123406ace0 msgr2=0x7f123406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:52.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f123406ace0 0x7f123406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1244000f60 tx=0x7f1244007ed0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1250156ab0 msgr2=0x7f1250154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:52.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1250156ab0 0x7f1250154c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f123c00e9b0 tx=0x7f123c00ecc0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 shutdown_connections 2023-07-06T02:22:52.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12501600b0 0x7f12500b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12500b19d0 0x7f12500ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f123406ace0 0x7f123406d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 --2- 172.21.15.89:0/2254351642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1250156ab0 0x7f1250154c40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.409+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 >> 172.21.15.89:0/2254351642 conn(0x7f12500cc490 msgr2=0x7f125015e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:52.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.410+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 shutdown_connections 2023-07-06T02:22:52.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.411+0000 7f1256fc4700 1 -- 172.21.15.89:0/2254351642 wait complete. 2023-07-06T02:22:52.421 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-07-06T02:22:52.432 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T02:22:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.554+0000 7f04cc783700 1 Processor -- start 2023-07-06T02:22:52.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.554+0000 7f04cc783700 1 -- start start 2023-07-06T02:22:52.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cc783700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f04c4156ab0 0x7f04c4160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:52.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cc783700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c41600b0 0x7f04c40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:52.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cc783700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c40cb3d0 con 0x7f04c4160bb0 2023-07-06T02:22:52.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cc783700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c4156e80 con 0x7f04c4156ab0 2023-07-06T02:22:52.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cc783700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40ca370 con 0x7f04c41600b0 2023-07-06T02:22:52.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04ca51f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f04c4156ab0 0x7f04c4160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:52.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cad20700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c41600b0 0x7f04c40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:52.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04c9d1e700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f04c4160bb0 0x7f04c40b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:60800/0 (socket says 172.21.15.89:60800) 2023-07-06T02:22:52.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cad20700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c41600b0 0x7f04c40cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55122/0 (socket says 172.21.15.89:55122) 2023-07-06T02:22:52.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cad20700 1 -- 172.21.15.89:0/3918726118 learned_addr learned my addr 172.21.15.89:0/3918726118 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:52.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04ca51f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f04c4156ab0 0x7f04c4160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:58900/0 (socket says 172.21.15.89:58900) 2023-07-06T02:22:52.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cad20700 1 -- 172.21.15.89:0/3918726118 >> v2:172.21.15.172:3300/0 conn(0x7f04c4156ab0 msgr2=0x7f04c4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:52.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cad20700 1 --2- 172.21.15.89:0/3918726118 >> v2:172.21.15.172:3300/0 conn(0x7f04c4156ab0 0x7f04c4160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cad20700 1 -- 172.21.15.89:0/3918726118 >> v1:172.21.15.172:6789/0 conn(0x7f04c4160bb0 legacy=0x7f04c40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:52.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04cad20700 1 -- 172.21.15.89:0/3918726118 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c40ca610 con 0x7f04c41600b0 2023-07-06T02:22:52.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.555+0000 7f04ca51f700 1 --2- 172.21.15.89:0/3918726118 >> v2:172.21.15.172:3300/0 conn(0x7f04c4156ab0 0x7f04c4160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:52.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cad20700 1 --2- 172.21.15.89:0/3918726118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c41600b0 0x7f04c40cd700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009e50 tx=0x7f04bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277ac12aba554194 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:52.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04c951d700 1 -- 172.21.15.89:0/3918726118 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00e070 con 0x7f04c41600b0 2023-07-06T02:22:52.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04c951d700 1 -- 172.21.15.89:0/3918726118 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc00b910 con 0x7f04c41600b0 2023-07-06T02:22:52.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04c951d700 1 -- 172.21.15.89:0/3918726118 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00bcc0 con 0x7f04c41600b0 2023-07-06T02:22:52.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 -- 172.21.15.89:0/3918726118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c41600b0 msgr2=0x7f04c40cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:52.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 --2- 172.21.15.89:0/3918726118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c41600b0 0x7f04c40cd700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009e50 tx=0x7f04bc00b330 comp rx=0 tx=0).stop 2023-07-06T02:22:52.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 -- 172.21.15.89:0/3918726118 shutdown_connections 2023-07-06T02:22:52.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 --2- 172.21.15.89:0/3918726118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c41600b0 0x7f04c40cd700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 --2- 172.21.15.89:0/3918726118 >> v2:172.21.15.172:3300/0 conn(0x7f04c4156ab0 0x7f04c4160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 -- 172.21.15.89:0/3918726118 >> 172.21.15.89:0/3918726118 conn(0x7f04c40cc490 msgr2=0x7f04c40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:52.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 -- 172.21.15.89:0/3918726118 shutdown_connections 2023-07-06T02:22:52.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.556+0000 7f04cc783700 1 -- 172.21.15.89:0/3918726118 wait complete. 2023-07-06T02:22:52.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 Processor -- start 2023-07-06T02:22:52.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 -- start start 2023-07-06T02:22:52.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c4156ab0 0x7f04c40ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:52.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c41600b0 0x7f04c40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:52.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c40c5880 0x7f04c40c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:52.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40b2f20 con 0x7f04c41600b0 2023-07-06T02:22:52.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c4150670 con 0x7f04c40c5880 2023-07-06T02:22:52.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cc783700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40ae750 con 0x7f04c4156ab0 2023-07-06T02:22:52.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04c9d1e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c41600b0 0x7f04c40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:52.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04c9d1e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c41600b0 0x7f04c40c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60150/0 (socket says 172.21.15.89:60150) 2023-07-06T02:22:52.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04ca51f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c4156ab0 0x7f04c40ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:52.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04c9d1e700 1 -- 172.21.15.89:0/1971858006 learned_addr learned my addr 172.21.15.89:0/1971858006 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:52.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.558+0000 7f04cad20700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c40c5880 0x7f04c40c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:52.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04c9d1e700 1 -- 172.21.15.89:0/1971858006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c4156ab0 msgr2=0x7f04c40ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:52.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04c9d1e700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c4156ab0 0x7f04c40ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04c9d1e700 1 -- 172.21.15.89:0/1971858006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c40c5880 msgr2=0x7f04c40c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:52.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04c9d1e700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c40c5880 0x7f04c40c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04c9d1e700 1 -- 172.21.15.89:0/1971858006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00094a0 con 0x7f04c41600b0 2023-07-06T02:22:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04ca51f700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c4156ab0 0x7f04c40ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:22:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04c9d1e700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c41600b0 0x7f04c40c5340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c530 tx=0x7f04b400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04baffd700 1 -- 172.21.15.89:0/1971858006 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4005cb0 con 0x7f04c41600b0 2023-07-06T02:22:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04baffd700 1 -- 172.21.15.89:0/1971858006 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4005e10 con 0x7f04c41600b0 2023-07-06T02:22:52.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc009b30 con 0x7f04c41600b0 2023-07-06T02:22:52.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04baffd700 1 -- 172.21.15.89:0/1971858006 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40072d0 con 0x7f04c41600b0 2023-07-06T02:22:52.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.559+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c406c4f0 con 0x7f04c41600b0 2023-07-06T02:22:52.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.560+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c40b3060 con 0x7f04c41600b0 2023-07-06T02:22:52.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.561+0000 7f04baffd700 1 -- 172.21.15.89:0/1971858006 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f04b4007430 con 0x7f04c41600b0 2023-07-06T02:22:52.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.561+0000 7f04baffd700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04a806ace0 0x7f04a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:52.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.561+0000 7f04ca51f700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04a806ace0 0x7f04a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:52.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.561+0000 7f04baffd700 1 -- 172.21.15.89:0/1971858006 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f04b4088590 con 0x7f04c41600b0 2023-07-06T02:22:52.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.561+0000 7f04ca51f700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04a806ace0 0x7f04a806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f04c406d220 tx=0x7f04c0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:52.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.565+0000 7f04baffd700 1 -- 172.21.15.89:0/1971858006 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f04c40b3060 con 0x7f04c41600b0 2023-07-06T02:22:52.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:52.796+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f04c40b3060 con 0x7f04c41600b0 2023-07-06T02:22:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.404+0000 7f04baffd700 1 -- 172.21.15.89:0/1971858006 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f04c40b3060 con 0x7f04c41600b0 2023-07-06T02:22:53.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.416+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04a806ace0 msgr2=0x7f04a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:53.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.416+0000 7f04cc783700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04a806ace0 0x7f04a806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f04c406d220 tx=0x7f04c0009210 comp rx=0 tx=0).stop 2023-07-06T02:22:53.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.416+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c41600b0 msgr2=0x7f04c40c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:53.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.416+0000 7f04cc783700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c41600b0 0x7f04c40c5340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c530 tx=0x7f04b400c840 comp rx=0 tx=0).stop 2023-07-06T02:22:53.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.418+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 shutdown_connections 2023-07-06T02:22:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.418+0000 7f04cc783700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c40c5880 0x7f04c40c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.418+0000 7f04cc783700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c4156ab0 0x7f04c40ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.418+0000 7f04cc783700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04a806ace0 0x7f04a806d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.418+0000 7f04cc783700 1 --2- 172.21.15.89:0/1971858006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c41600b0 0x7f04c40c5340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.418+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 >> 172.21.15.89:0/1971858006 conn(0x7f04c40cc490 msgr2=0x7f04c40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:53.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.421+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 shutdown_connections 2023-07-06T02:22:53.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.421+0000 7f04cc783700 1 -- 172.21.15.89:0/1971858006 wait complete. 2023-07-06T02:22:53.431 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 3 and data pool 4 2023-07-06T02:22:53.447 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-06T02:22:53.448 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-06T02:22:53.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.573+0000 7fb0c1c72700 1 Processor -- start 2023-07-06T02:22:53.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.574+0000 7fb0c1c72700 1 -- start start 2023-07-06T02:22:53.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.574+0000 7fb0c1c72700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 0x7fb0bc071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.574+0000 7fb0c1c72700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc072000 con 0x7fb0bc156c30 2023-07-06T02:22:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.574+0000 7fb0c1c72700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc157000 con 0x7fb0bc0ca0c0 2023-07-06T02:22:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.574+0000 7fb0c1c72700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0cdd00 con 0x7fb0bc071670 2023-07-06T02:22:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.574+0000 7fb0bb7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb0bc156c30 0x7fb0bc0cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56838/0 (socket says 172.21.15.89:56838) 2023-07-06T02:22:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.574+0000 7fb0bb7fe700 1 -- 172.21.15.89:0/3049308008 learned_addr learned my addr 172.21.15.89:0/3049308008 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0baffd700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 0x7fb0bc071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239563282 0 0) 0x7fb0bc072000 con 0x7fb0bc156c30 2023-07-06T02:22:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a80035b0 con 0x7fb0bc156c30 2023-07-06T02:22:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911275758 0 0) 0x7fb0bc157000 con 0x7fb0bc0ca0c0 2023-07-06T02:22:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0bc072000 con 0x7fb0bc0ca0c0 2023-07-06T02:22:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35244107 0 0) 0x7fb0a80035b0 con 0x7fb0bc156c30 2023-07-06T02:22:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 msgr2=0x7fb0bc071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 0x7fb0bc071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 >> v1:172.21.15.172:6789/0 conn(0x7fb0bc0ca0c0 legacy=0x7fb0bc0ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc1602c0 con 0x7fb0bc156c30 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fb0a40015f0 con 0x7fb0bc156c30 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 >> v1:172.21.15.89:6789/0 conn(0x7fb0bc156c30 legacy=0x7fb0bc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a8003bc0 0x7fb0a8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 0x7fb0a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0cdd00 con 0x7fb0a8004850 2023-07-06T02:22:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a800baf0 con 0x7fb0a8003bc0 2023-07-06T02:22:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a800bcb0 con 0x7fb0bc071670 2023-07-06T02:22:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0baffd700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0bb7fe700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a8003bc0 0x7fb0a8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.575+0000 7fb0bbfff700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 0x7fb0a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0baffd700 1 -- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 msgr2=0x7fb0a8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0baffd700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc071670 0x7fb0a8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0baffd700 1 -- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a8003bc0 msgr2=0x7fb0a8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0baffd700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a8003bc0 0x7fb0a8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0baffd700 1 -- 172.21.15.89:0/3049308008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80035b0 con 0x7fb0a8004850 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0baffd700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0a8004cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009b30 tx=0x7fb0a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d20f164470c9415 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00e6c0 con 0x7fb0a8004850 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac00e820 con 0x7fb0a8004850 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0ba7fc700 1 -- 172.21.15.89:0/3049308008 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00eb90 con 0x7fb0a8004850 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0c1c72700 1 -- 172.21.15.89:0/3049308008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 msgr2=0x7fb0a8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0c1c72700 1 --2- 172.21.15.89:0/3049308008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0a8004cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009b30 tx=0x7fb0a8005880 comp rx=0 tx=0).stop 2023-07-06T02:22:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0bbfff700 1 -- 172.21.15.89:0/3049308008 reap_dead start 2023-07-06T02:22:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0c1c72700 1 -- 172.21.15.89:0/3049308008 shutdown_connections 2023-07-06T02:22:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0c1c72700 1 -- 172.21.15.89:0/3049308008 >> 172.21.15.89:0/3049308008 conn(0x7fb0bc0cf970 msgr2=0x7fb0bc0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0c1c72700 1 -- 172.21.15.89:0/3049308008 shutdown_connections 2023-07-06T02:22:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.576+0000 7fb0c1c72700 1 -- 172.21.15.89:0/3049308008 wait complete. 2023-07-06T02:22:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.578+0000 7fb0c1c72700 1 Processor -- start 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.578+0000 7fb0c1c72700 1 -- start start 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.578+0000 7fb0c1c72700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0bc0c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.578+0000 7fb0c1c72700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc14c5b0 0x7fb0bc0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.578+0000 7fb0c1c72700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc0b4020 0x7fb0bc0ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.578+0000 7fb0c1c72700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0c4e70 con 0x7fb0a8004850 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.578+0000 7fb0c1c72700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0c4b90 con 0x7fb0bc0b4020 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0c1c72700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0aba90 con 0x7fb0bc14c5b0 2023-07-06T02:22:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0bc0c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0bc0c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60180/0 (socket says 172.21.15.89:60180) 2023-07-06T02:22:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 -- 172.21.15.89:0/645709120 learned_addr learned my addr 172.21.15.89:0/645709120 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0baffd700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc14c5b0 0x7fb0bc0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bbfff700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc0b4020 0x7fb0bc0ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 -- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc14c5b0 msgr2=0x7fb0bc0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc14c5b0 0x7fb0bc0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 -- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc0b4020 msgr2=0x7fb0bc0ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:53.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc0b4020 0x7fb0bc0ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:53.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 -- 172.21.15.89:0/645709120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0abd70 con 0x7fb0a8004850 2023-07-06T02:22:53.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bbfff700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc0b4020 0x7fb0bc0ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:22:53.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0bb7fe700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0bc0c5ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4007290 tx=0x7fb0a40075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:53.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0b3fff700 1 -- 172.21.15.89:0/645709120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400f6c0 con 0x7fb0a8004850 2023-07-06T02:22:53.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0b3fff700 1 -- 172.21.15.89:0/645709120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a400f820 con 0x7fb0a8004850 2023-07-06T02:22:53.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc0ac050 con 0x7fb0a8004850 2023-07-06T02:22:53.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0b3fff700 1 -- 172.21.15.89:0/645709120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400fbd0 con 0x7fb0a8004850 2023-07-06T02:22:53.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.579+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc14ca20 con 0x7fb0a8004850 2023-07-06T02:22:53.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.580+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc0ab8f0 con 0x7fb0a8004850 2023-07-06T02:22:53.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.580+0000 7fb0b3fff700 1 -- 172.21.15.89:0/645709120 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0a400fd30 con 0x7fb0a8004850 2023-07-06T02:22:53.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.581+0000 7fb0b3fff700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb09c06ace0 0x7fb09c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:53.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.581+0000 7fb0b3fff700 1 -- 172.21.15.89:0/645709120 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0a40892b0 con 0x7fb0a8004850 2023-07-06T02:22:53.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.581+0000 7fb0baffd700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb09c06ace0 0x7fb09c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:53.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.581+0000 7fb0baffd700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb09c06ace0 0x7fb09c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0090b0 tx=0x7fb0ac01f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:53.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.585+0000 7fb0b3fff700 1 -- 172.21.15.89:0/645709120 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb0bc0ab8f0 con 0x7fb0a8004850 2023-07-06T02:22:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:53.785+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb0bc160860 con 0x7fb0a8004850 2023-07-06T02:22:54.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.424+0000 7fb0b3fff700 1 -- 172.21.15.89:0/645709120 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb0bc160860 con 0x7fb0a8004850 2023-07-06T02:22:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb09c06ace0 msgr2=0x7fb09c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb09c06ace0 0x7fb09c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0090b0 tx=0x7fb0ac01f040 comp rx=0 tx=0).stop 2023-07-06T02:22:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 msgr2=0x7fb0bc0c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0bc0c5ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4007290 tx=0x7fb0a40075a0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 shutdown_connections 2023-07-06T02:22:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0bc0b4020 0x7fb0bc0ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb09c06ace0 0x7fb09c06d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0a8004850 0x7fb0bc0c5ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 --2- 172.21.15.89:0/645709120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0bc14c5b0 0x7fb0bc0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 >> 172.21.15.89:0/645709120 conn(0x7fb0bc0cf970 msgr2=0x7fb0bc06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 shutdown_connections 2023-07-06T02:22:54.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.427+0000 7fb0c1c72700 1 -- 172.21.15.89:0/645709120 wait complete. 2023-07-06T02:22:54.445 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T02:22:54.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.577+0000 7f0167d5c700 1 Processor -- start 2023-07-06T02:22:54.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.577+0000 7f0167d5c700 1 -- start start 2023-07-06T02:22:54.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0167d5c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0160156480 0x7f0160070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0167d5c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 0x7f01600cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0167d5c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01600cda60 con 0x7f01600cd5e0 2023-07-06T02:22:54.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0167d5c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0160159d90 con 0x7f0160156480 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0167d5c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0160156850 con 0x7f0160071400 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f01662f9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f01600cd5e0 0x7f016015c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56858/0 (socket says 172.21.15.89:56858) 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f01662f9700 1 -- 172.21.15.89:0/665409050 learned_addr learned my addr 172.21.15.89:0/665409050 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277179708 0 0) 0x7f01600cda60 con 0x7f01600cd5e0 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f014c0035b0 con 0x7f01600cd5e0 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0165af8700 1 --2- 172.21.15.89:0/665409050 >> v2:172.21.15.172:3300/0 conn(0x7f0160156480 0x7f0160070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f01652f7700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 0x7f01600cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2786356383 0 0) 0x7f014c0035b0 con 0x7f01600cd5e0 2023-07-06T02:22:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 msgr2=0x7f01600cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 0x7f01600cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 >> v2:172.21.15.172:3300/0 conn(0x7f0160156480 msgr2=0x7f0160070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 --2- 172.21.15.89:0/665409050 >> v2:172.21.15.172:3300/0 conn(0x7f0160156480 0x7f0160070ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016006e300 con 0x7f01600cd5e0 2023-07-06T02:22:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f01580019f0 con 0x7f01600cd5e0 2023-07-06T02:22:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 >> v1:172.21.15.89:6789/0 conn(0x7f01600cd5e0 legacy=0x7f016015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 0x7f014c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c003f80 0x7f014c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014c004d90 0x7f014c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0160159d90 con 0x7f014c004d90 2023-07-06T02:22:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.578+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0160156850 con 0x7f014c003f80 2023-07-06T02:22:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c00cb20 con 0x7f0160071400 2023-07-06T02:22:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01662f9700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014c004d90 0x7f014c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f0165af8700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 0x7f014c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01652f7700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c003f80 0x7f014c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01662f9700 1 -- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 msgr2=0x7f014c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01662f9700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0160071400 0x7f014c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01662f9700 1 -- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c003f80 msgr2=0x7f014c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01662f9700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c003f80 0x7f014c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01662f9700 1 -- 172.21.15.89:0/665409050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0035b0 con 0x7f014c004d90 2023-07-06T02:22:54.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f01662f9700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014c004d90 0x7f014c005200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f015800a8d0 tx=0x7f014c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c1f4bb984e0a22 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015800a340 con 0x7f014c004d90 2023-07-06T02:22:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0158012600 con 0x7f014c004d90 2023-07-06T02:22:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.579+0000 7f0164af6700 1 -- 172.21.15.89:0/665409050 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01580129b0 con 0x7f014c004d90 2023-07-06T02:22:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.580+0000 7f0167d5c700 1 -- 172.21.15.89:0/665409050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014c004d90 msgr2=0x7f014c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.580+0000 7f0167d5c700 1 --2- 172.21.15.89:0/665409050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014c004d90 0x7f014c005200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f015800a8d0 tx=0x7f014c005880 comp rx=0 tx=0).stop 2023-07-06T02:22:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.580+0000 7f01662f9700 1 -- 172.21.15.89:0/665409050 reap_dead start 2023-07-06T02:22:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.580+0000 7f0167d5c700 1 -- 172.21.15.89:0/665409050 shutdown_connections 2023-07-06T02:22:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.580+0000 7f0167d5c700 1 -- 172.21.15.89:0/665409050 >> 172.21.15.89:0/665409050 conn(0x7f01600cbb90 msgr2=0x7f01600cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.580+0000 7f0167d5c700 1 -- 172.21.15.89:0/665409050 shutdown_connections 2023-07-06T02:22:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.580+0000 7f0167d5c700 1 -- 172.21.15.89:0/665409050 wait complete. 2023-07-06T02:22:54.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.581+0000 7f0167d5c700 1 Processor -- start 2023-07-06T02:22:54.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.581+0000 7f0167d5c700 1 -- start start 2023-07-06T02:22:54.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c004d90 0x7f016014f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f016015a920 0x7f01600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01600ca8f0 0x7f01600cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01600cb2d0 con 0x7f01600ca8f0 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016015b9f0 con 0x7f014c004d90 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016015ade0 con 0x7f016015a920 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01600ca8f0 0x7f01600cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01600ca8f0 0x7f01600cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60202/0 (socket says 172.21.15.89:60202) 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 -- 172.21.15.89:0/751458246 learned_addr learned my addr 172.21.15.89:0/751458246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:54.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0165af8700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c004d90 0x7f016014f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01652f7700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f016015a920 0x7f01600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 -- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f016015a920 msgr2=0x7f01600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:54.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f016015a920 0x7f01600b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 -- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c004d90 msgr2=0x7f016014f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:54.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c004d90 0x7f016014f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:54.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 -- 172.21.15.89:0/751458246 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016015b0c0 con 0x7f01600ca8f0 2023-07-06T02:22:54.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01662f9700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01600ca8f0 0x7f01600cad90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0158012000 tx=0x7f0158001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01567fc700 1 -- 172.21.15.89:0/751458246 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015800a340 con 0x7f01600ca8f0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01567fc700 1 -- 172.21.15.89:0/751458246 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01580019c0 con 0x7f01600ca8f0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f01567fc700 1 -- 172.21.15.89:0/751458246 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015801e640 con 0x7f01600ca8f0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016015b3a0 con 0x7f01600ca8f0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.582+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016014f830 con 0x7f01600ca8f0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.583+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0160074270 con 0x7f01600ca8f0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.584+0000 7f01567fc700 1 -- 172.21.15.89:0/751458246 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0158019030 con 0x7f01600ca8f0 2023-07-06T02:22:54.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.584+0000 7f01567fc700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f014406ace0 0x7f014406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:54.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.584+0000 7f01567fc700 1 -- 172.21.15.89:0/751458246 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f015808ad60 con 0x7f01600ca8f0 2023-07-06T02:22:54.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.584+0000 7f0165af8700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f014406ace0 0x7f014406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:54.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.584+0000 7f0165af8700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f014406ace0 0x7f014406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f015c009ce0 tx=0x7f015c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:54.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.588+0000 7f01567fc700 1 -- 172.21.15.89:0/751458246 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0158055fa0 con 0x7f01600ca8f0 2023-07-06T02:22:54.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:54.775+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f01600cb4c0 con 0x7f01600ca8f0 2023-07-06T02:22:56.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.429+0000 7f01567fc700 1 -- 172.21.15.89:0/751458246 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f0158017020 con 0x7f01600ca8f0 2023-07-06T02:22:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f014406ace0 msgr2=0x7f014406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f014406ace0 0x7f014406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f015c009ce0 tx=0x7f015c009210 comp rx=0 tx=0).stop 2023-07-06T02:22:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01600ca8f0 msgr2=0x7f01600cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:56.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01600ca8f0 0x7f01600cad90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0158012000 tx=0x7f0158001360 comp rx=0 tx=0).stop 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 shutdown_connections 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f014406ace0 0x7f014406d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01600ca8f0 0x7f01600cad90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f014c004d90 0x7f016014f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 --2- 172.21.15.89:0/751458246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f016015a920 0x7f01600b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 >> 172.21.15.89:0/751458246 conn(0x7f01600cbb90 msgr2=0x7f016006a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 shutdown_connections 2023-07-06T02:22:56.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.435+0000 7f0167d5c700 1 -- 172.21.15.89:0/751458246 wait complete. 2023-07-06T02:22:56.441 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' created 2023-07-06T02:22:56.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-06T02:22:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.583+0000 7fdd508e2700 1 Processor -- start 2023-07-06T02:22:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.583+0000 7fdd508e2700 1 -- start start 2023-07-06T02:22:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd508e2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdd4c159030 0x7fdd4c071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd508e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd4c0ceb10 0x7fdd4c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd508e2700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c15daa0 con 0x7fdd4c159400 2023-07-06T02:22:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd508e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c15ced0 con 0x7fdd4c0ceb10 2023-07-06T02:22:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd508e2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c15d170 con 0x7fdd4c159030 2023-07-06T02:22:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd4ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd4c0ceb10 0x7fdd4c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd4ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd4c0ceb10 0x7fdd4c071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60210/0 (socket says 172.21.15.89:60210) 2023-07-06T02:22:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/3477058844 learned_addr learned my addr 172.21.15.89:0/3477058844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.584+0000 7fdd4a59c700 1 --2- 172.21.15.89:0/3477058844 >> v2:172.21.15.172:3300/0 conn(0x7fdd4c159030 0x7fdd4c071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/3477058844 >> v2:172.21.15.172:3300/0 conn(0x7fdd4c159030 msgr2=0x7fdd4c071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/3477058844 >> v2:172.21.15.172:3300/0 conn(0x7fdd4c159030 0x7fdd4c071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/3477058844 >> v1:172.21.15.172:6789/0 conn(0x7fdd4c159400 legacy=0x7fdd4c0ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4ad9d700 1 -- 172.21.15.89:0/3477058844 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c15d450 con 0x7fdd4c0ceb10 2023-07-06T02:22:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/3477058844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c0ceb10 0x7fdd4c071bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0b2540 tx=0x7fdd4400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407724581a0ad229 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4959a700 1 -- 172.21.15.89:0/3477058844 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44010070 con 0x7fdd4c0ceb10 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4959a700 1 -- 172.21.15.89:0/3477058844 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4400dce0 con 0x7fdd4c0ceb10 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd4959a700 1 -- 172.21.15.89:0/3477058844 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400b310 con 0x7fdd4c0ceb10 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd508e2700 1 -- 172.21.15.89:0/3477058844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c0ceb10 msgr2=0x7fdd4c071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.585+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3477058844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c0ceb10 0x7fdd4c071bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0b2540 tx=0x7fdd4400d430 comp rx=0 tx=0).stop 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.586+0000 7fdd508e2700 1 -- 172.21.15.89:0/3477058844 shutdown_connections 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.586+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3477058844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c0ceb10 0x7fdd4c071bb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.586+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3477058844 >> v2:172.21.15.172:3300/0 conn(0x7fdd4c159030 0x7fdd4c071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.586+0000 7fdd508e2700 1 -- 172.21.15.89:0/3477058844 >> 172.21.15.89:0/3477058844 conn(0x7fdd4c0cf970 msgr2=0x7fdd4c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.586+0000 7fdd508e2700 1 -- 172.21.15.89:0/3477058844 shutdown_connections 2023-07-06T02:22:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.586+0000 7fdd508e2700 1 -- 172.21.15.89:0/3477058844 wait complete. 2023-07-06T02:22:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.587+0000 7fdd508e2700 1 Processor -- start 2023-07-06T02:22:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.587+0000 7fdd508e2700 1 -- start start 2023-07-06T02:22:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd4c0ceb10 0x7fdd4c0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c159030 0x7fdd4c06c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd4c159400 0x7fdd4c0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c0b0ca0 con 0x7fdd4c159030 2023-07-06T02:22:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1530d0 con 0x7fdd4c159400 2023-07-06T02:22:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c0c7de0 con 0x7fdd4c0ceb10 2023-07-06T02:22:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c159030 0x7fdd4c06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c159030 0x7fdd4c06c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60214/0 (socket says 172.21.15.89:60214) 2023-07-06T02:22:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 -- 172.21.15.89:0/3210075441 learned_addr learned my addr 172.21.15.89:0/3210075441 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd4a59c700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd4c0ceb10 0x7fdd4c0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd4c159400 0x7fdd4c0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 -- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd4c0ceb10 msgr2=0x7fdd4c0b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd4c0ceb10 0x7fdd4c0b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 -- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd4c159400 msgr2=0x7fdd4c0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd4c159400 0x7fdd4c0b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 -- 172.21.15.89:0/3210075441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4400bd40 con 0x7fdd4c159030 2023-07-06T02:22:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd4ad9d700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd4c159400 0x7fdd4c0b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd49d9b700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c159030 0x7fdd4c06c790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c420 tx=0x7fdd3c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd3affd700 1 -- 172.21.15.89:0/3210075441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0052d0 con 0x7fdd4c159030 2023-07-06T02:22:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd3affd700 1 -- 172.21.15.89:0/3210075441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c005430 con 0x7fdd4c159030 2023-07-06T02:22:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c0c80c0 con 0x7fdd4c159030 2023-07-06T02:22:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd3affd700 1 -- 172.21.15.89:0/3210075441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0038c0 con 0x7fdd4c159030 2023-07-06T02:22:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.588+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c0c8680 con 0x7fdd4c159030 2023-07-06T02:22:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.589+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c0742e0 con 0x7fdd4c159030 2023-07-06T02:22:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.590+0000 7fdd3affd700 1 -- 172.21.15.89:0/3210075441 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd3c003a20 con 0x7fdd4c159030 2023-07-06T02:22:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.590+0000 7fdd3affd700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdd2c06ace0 0x7fdd2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.590+0000 7fdd3affd700 1 -- 172.21.15.89:0/3210075441 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fdd3c088b70 con 0x7fdd4c159030 2023-07-06T02:22:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.590+0000 7fdd4a59c700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdd2c06ace0 0x7fdd2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.591+0000 7fdd4a59c700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdd2c06ace0 0x7fdd2c06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009b70 tx=0x7fdd40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.595+0000 7fdd3affd700 1 -- 172.21.15.89:0/3210075441 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd3c053960 con 0x7fdd4c159030 2023-07-06T02:22:56.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:56.781+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fdd4c06bc80 con 0x7fdd4c159030 2023-07-06T02:22:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.432+0000 7fdd3affd700 1 -- 172.21.15.89:0/3210075441 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdd3c056f80 con 0x7fdd4c159030 2023-07-06T02:22:57.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdd2c06ace0 msgr2=0x7fdd2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:57.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdd2c06ace0 0x7fdd2c06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009b70 tx=0x7fdd40008040 comp rx=0 tx=0).stop 2023-07-06T02:22:57.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c159030 msgr2=0x7fdd4c06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:57.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c159030 0x7fdd4c06c790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c420 tx=0x7fdd3c00c7e0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 shutdown_connections 2023-07-06T02:22:57.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd4c159400 0x7fdd4c0b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd4c0ceb10 0x7fdd4c0b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdd2c06ace0 0x7fdd2c06d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 --2- 172.21.15.89:0/3210075441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd4c159030 0x7fdd4c06c790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.440+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 >> 172.21.15.89:0/3210075441 conn(0x7fdd4c0cf970 msgr2=0x7fdd4c071570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:57.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.441+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 shutdown_connections 2023-07-06T02:22:57.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.441+0000 7fdd508e2700 1 -- 172.21.15.89:0/3210075441 wait complete. 2023-07-06T02:22:57.450 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 5 allow_ec_overwrites to true 2023-07-06T02:22:57.468 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-06T02:22:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.590+0000 7ff5b29c3700 1 Processor -- start 2023-07-06T02:22:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.590+0000 7ff5b29c3700 1 -- start start 2023-07-06T02:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5b29c3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 0x7ff5ac0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5b29c3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5ac06dc10 0x7ff5ac157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5b29c3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac06e110 con 0x7ff5ac0cd180 2023-07-06T02:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5b29c3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac06e3f0 con 0x7ff5ac06dc10 2023-07-06T02:22:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5b29c3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac06e6d0 con 0x7ff5ac0705a0 2023-07-06T02:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5ab7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff5ac0cd180 0x7ff5ac15a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56890/0 (socket says 172.21.15.89:56890) 2023-07-06T02:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/4290884393 learned_addr learned my addr 172.21.15.89:0/4290884393 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5abfff700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 0x7ff5ac0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5b0f60700 1 --2- 172.21.15.89:0/4290884393 >> v2:172.21.15.172:3300/0 conn(0x7ff5ac06dc10 0x7ff5ac157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714944269 0 0) 0x7ff5ac06e110 con 0x7ff5ac0cd180 2023-07-06T02:22:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5940035b0 con 0x7ff5ac0cd180 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2385651853 0 0) 0x7ff5940035b0 con 0x7ff5ac0cd180 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 msgr2=0x7ff5ac0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 0x7ff5ac0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 >> v2:172.21.15.172:3300/0 conn(0x7ff5ac06dc10 msgr2=0x7ff5ac157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/4290884393 >> v2:172.21.15.172:3300/0 conn(0x7ff5ac06dc10 0x7ff5ac157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac06e9b0 con 0x7ff5ac0cd180 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7ff59c001670 con 0x7ff5ac0cd180 2023-07-06T02:22:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 >> v1:172.21.15.89:6789/0 conn(0x7ff5ac0cd180 legacy=0x7ff5ac15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 0x7ff594003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff594003f80 0x7ff594004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff594004d90 0x7ff594005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac06e3f0 con 0x7ff594003f80 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac06e6d0 con 0x7ff594004d90 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59400ba10 con 0x7ff5ac0705a0 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.591+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff594003f80 0x7ff594004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5abfff700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 0x7ff594003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5b0f60700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff594004d90 0x7ff594005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 msgr2=0x7ff594003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ac0705a0 0x7ff594003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff594004d90 msgr2=0x7ff594005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff594004d90 0x7ff594005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/4290884393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940035b0 con 0x7ff594003f80 2023-07-06T02:22:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff594003f80 0x7ff594004850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0cd550 tx=0x7ff594005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b999a9f548668d09 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c007c60 con 0x7ff594003f80 2023-07-06T02:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c00ce80 con 0x7ff594003f80 2023-07-06T02:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5aaffd700 1 -- 172.21.15.89:0/4290884393 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c005ba0 con 0x7ff594003f80 2023-07-06T02:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5b29c3700 1 -- 172.21.15.89:0/4290884393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff594003f80 msgr2=0x7ff594004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5b29c3700 1 --2- 172.21.15.89:0/4290884393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff594003f80 0x7ff594004850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0cd550 tx=0x7ff594005880 comp rx=0 tx=0).stop 2023-07-06T02:22:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5b0f60700 1 -- 172.21.15.89:0/4290884393 reap_dead start 2023-07-06T02:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5b29c3700 1 -- 172.21.15.89:0/4290884393 shutdown_connections 2023-07-06T02:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.592+0000 7ff5b29c3700 1 -- 172.21.15.89:0/4290884393 >> 172.21.15.89:0/4290884393 conn(0x7ff5ac0d0b00 msgr2=0x7ff5ac0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.593+0000 7ff5b29c3700 1 -- 172.21.15.89:0/4290884393 shutdown_connections 2023-07-06T02:22:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.593+0000 7ff5b29c3700 1 -- 172.21.15.89:0/4290884393 wait complete. 2023-07-06T02:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.594+0000 7ff5b29c3700 1 Processor -- start 2023-07-06T02:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.594+0000 7ff5b29c3700 1 -- start start 2023-07-06T02:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff594003f80 0x7ff5ac1ef880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac0cf330 0x7ff5ac0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ac0c9ea0 0x7ff5ac0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac145350 con 0x7ff5ac0cf330 2023-07-06T02:22:57.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0cfce0 con 0x7ff5ac0c9ea0 2023-07-06T02:22:57.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac144a40 con 0x7ff594003f80 2023-07-06T02:22:57.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac0cf330 0x7ff5ac0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac0cf330 0x7ff5ac0cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60232/0 (socket says 172.21.15.89:60232) 2023-07-06T02:22:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/2544165591 learned_addr learned my addr 172.21.15.89:0/2544165591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5abfff700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff594003f80 0x7ff5ac1ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b0f60700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ac0c9ea0 0x7ff5ac0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff594003f80 msgr2=0x7ff5ac1ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff594003f80 0x7ff5ac1ef880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ac0c9ea0 msgr2=0x7ff5ac0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ac0c9ea0 0x7ff5ac0ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 -- 172.21.15.89:0/2544165591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac1478b0 con 0x7ff5ac0cf330 2023-07-06T02:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5ab7fe700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac0cf330 0x7ff5ac0cf7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5940046d0 tx=0x7ff59c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5a8ff9700 1 -- 172.21.15.89:0/2544165591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0072e0 con 0x7ff5ac0cf330 2023-07-06T02:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5a8ff9700 1 -- 172.21.15.89:0/2544165591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c007460 con 0x7ff5ac0cf330 2023-07-06T02:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac0cabb0 con 0x7ff5ac0cf330 2023-07-06T02:22:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5a8ff9700 1 -- 172.21.15.89:0/2544165591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c007810 con 0x7ff5ac0cf330 2023-07-06T02:22:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.595+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac144f70 con 0x7ff5ac0cf330 2023-07-06T02:22:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.596+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac145490 con 0x7ff5ac0cf330 2023-07-06T02:22:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.597+0000 7ff5a8ff9700 1 -- 172.21.15.89:0/2544165591 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff59c004d50 con 0x7ff5ac0cf330 2023-07-06T02:22:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.597+0000 7ff5a8ff9700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff59806ace0 0x7ff59806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.597+0000 7ff5a8ff9700 1 -- 172.21.15.89:0/2544165591 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7ff59c089cf0 con 0x7ff5ac0cf330 2023-07-06T02:22:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.597+0000 7ff5abfff700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff59806ace0 0x7ff59806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.597+0000 7ff5abfff700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff59806ace0 0x7ff59806d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00099d0 tx=0x7ff5a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.601+0000 7ff5a8ff9700 1 -- 172.21.15.89:0/2544165591 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff5ac145490 con 0x7ff5ac0cf330 2023-07-06T02:22:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:57.836+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7ff5ac074270 con 0x7ff5ac0cf330 2023-07-06T02:22:58.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:22:58.130 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:22:58.459 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-07-06T02:22:58.457+0000 7fc1fcf4e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T02:22:58.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.466+0000 7ff5a8ff9700 1 -- 172.21.15.89:0/2544165591 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7ff59c054b60 con 0x7ff5ac0cf330 2023-07-06T02:22:58.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff59806ace0 msgr2=0x7ff59806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff59806ace0 0x7ff59806d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00099d0 tx=0x7ff5a0008040 comp rx=0 tx=0).stop 2023-07-06T02:22:58.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac0cf330 msgr2=0x7ff5ac0cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac0cf330 0x7ff5ac0cf7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5940046d0 tx=0x7ff59c000fe0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 shutdown_connections 2023-07-06T02:22:58.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ac0c9ea0 0x7ff5ac0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff594003f80 0x7ff5ac1ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff59806ace0 0x7ff59806d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 --2- 172.21.15.89:0/2544165591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5ac0cf330 0x7ff5ac0cf7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 >> 172.21.15.89:0/2544165591 conn(0x7ff5ac0d0b00 msgr2=0x7ff5ac0ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:58.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 shutdown_connections 2023-07-06T02:22:58.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.468+0000 7ff5b29c3700 1 -- 172.21.15.89:0/2544165591 wait complete. 2023-07-06T02:22:58.474 INFO:teuthology.orchestra.run.smithi089.stderr:added data pool 5 to fsmap 2023-07-06T02:22:58.483 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:22:58.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.603+0000 7efddc773700 1 Processor -- start 2023-07-06T02:22:58.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.603+0000 7efddc773700 1 -- start start 2023-07-06T02:22:58.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efddc773700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd415c500 0x7efdd406edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efddc773700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efdd4070f20 0x7efdd4071390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efddc773700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd40cb9f0 con 0x7efdd4160000 2023-07-06T02:22:58.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efddc773700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd40718d0 con 0x7efdd4070f20 2023-07-06T02:22:58.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efddc773700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd415d8e0 con 0x7efdd415c500 2023-07-06T02:22:58.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efdd9d0e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efdd4070f20 0x7efdd4071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efdd9d0e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efdd4070f20 0x7efdd4071390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60238/0 (socket says 172.21.15.89:60238) 2023-07-06T02:22:58.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efdd9d0e700 1 -- 172.21.15.89:0/2754409326 learned_addr learned my addr 172.21.15.89:0/2754409326 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:58.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efdda50f700 1 --2- 172.21.15.89:0/2754409326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd415c500 0x7efdd406edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efdd9d0e700 1 -- 172.21.15.89:0/2754409326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd415c500 msgr2=0x7efdd406edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:58.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efdd9d0e700 1 --2- 172.21.15.89:0/2754409326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd415c500 0x7efdd406edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.604+0000 7efdd9d0e700 1 -- 172.21.15.89:0/2754409326 >> v1:172.21.15.172:6789/0 conn(0x7efdd4160000 legacy=0x7efdd40ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:58.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efdd9d0e700 1 -- 172.21.15.89:0/2754409326 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd406f310 con 0x7efdd4070f20 2023-07-06T02:22:58.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efdda50f700 1 --2- 172.21.15.89:0/2754409326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd415c500 0x7efdd406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:58.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efdd9d0e700 1 --2- 172.21.15.89:0/2754409326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd4070f20 0x7efdd4071390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdd00098f0 tx=0x7efdd0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb0f565d6bbdc4c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:58.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efdd950d700 1 -- 172.21.15.89:0/2754409326 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0004230 con 0x7efdd4070f20 2023-07-06T02:22:58.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efdd950d700 1 -- 172.21.15.89:0/2754409326 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0004390 con 0x7efdd4070f20 2023-07-06T02:22:58.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efdd950d700 1 -- 172.21.15.89:0/2754409326 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00036a0 con 0x7efdd4070f20 2023-07-06T02:22:58.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 -- 172.21.15.89:0/2754409326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd4070f20 msgr2=0x7efdd4071390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 --2- 172.21.15.89:0/2754409326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd4070f20 0x7efdd4071390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdd00098f0 tx=0x7efdd0009c00 comp rx=0 tx=0).stop 2023-07-06T02:22:58.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 -- 172.21.15.89:0/2754409326 shutdown_connections 2023-07-06T02:22:58.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 --2- 172.21.15.89:0/2754409326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd415c500 0x7efdd406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 --2- 172.21.15.89:0/2754409326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd4070f20 0x7efdd4071390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 -- 172.21.15.89:0/2754409326 >> 172.21.15.89:0/2754409326 conn(0x7efdd40cf1c0 msgr2=0x7efdd40cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:58.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 -- 172.21.15.89:0/2754409326 shutdown_connections 2023-07-06T02:22:58.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.605+0000 7efddc773700 1 -- 172.21.15.89:0/2754409326 wait complete. 2023-07-06T02:22:58.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.607+0000 7efddc773700 1 Processor -- start 2023-07-06T02:22:58.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.607+0000 7efddc773700 1 -- start start 2023-07-06T02:22:58.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efddc773700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd4070f20 0x7efdd40c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efddc773700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd415c500 0x7efdd4159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efddc773700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd4160000 0x7efdd4158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efddc773700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd40c49f0 con 0x7efdd415c500 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efddc773700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd4159180 con 0x7efdd4070f20 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efddc773700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd40c3700 con 0x7efdd4160000 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd415c500 0x7efdd4159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd415c500 0x7efdd4159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60246/0 (socket says 172.21.15.89:60246) 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 -- 172.21.15.89:0/1397496046 learned_addr learned my addr 172.21.15.89:0/1397496046 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:58.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdda50f700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd4070f20 0x7efdd40c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efddad10700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd4160000 0x7efdd4158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 -- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd4160000 msgr2=0x7efdd4158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd4160000 0x7efdd4158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 -- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd4070f20 msgr2=0x7efdd40c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd4070f20 0x7efdd40c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 -- 172.21.15.89:0/1397496046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0009550 con 0x7efdd415c500 2023-07-06T02:22:58.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdd9d0e700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd415c500 0x7efdd4159320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdd4066d50 tx=0x7efdd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:58.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.608+0000 7efdcaffd700 1 -- 172.21.15.89:0/1397496046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0003e30 con 0x7efdd415c500 2023-07-06T02:22:58.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.609+0000 7efdcaffd700 1 -- 172.21.15.89:0/1397496046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd001f070 con 0x7efdd415c500 2023-07-06T02:22:58.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.609+0000 7efdcaffd700 1 -- 172.21.15.89:0/1397496046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000b8a0 con 0x7efdd415c500 2023-07-06T02:22:58.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.609+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd40c4560 con 0x7efdd415c500 2023-07-06T02:22:58.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.609+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd40c3c30 con 0x7efdd415c500 2023-07-06T02:22:58.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.610+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd415dbe0 con 0x7efdd415c500 2023-07-06T02:22:58.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.610+0000 7efdcaffd700 1 -- 172.21.15.89:0/1397496046 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efdd0021030 con 0x7efdd415c500 2023-07-06T02:22:58.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.610+0000 7efdcaffd700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efdb806ace0 0x7efdb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.610+0000 7efdcaffd700 1 -- 172.21.15.89:0/1397496046 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efdd008add0 con 0x7efdd415c500 2023-07-06T02:22:58.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.610+0000 7efdda50f700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efdb806ace0 0x7efdb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.611+0000 7efdda50f700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efdb806ace0 0x7efdb806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006fd0 tx=0x7efdc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:58.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.615+0000 7efdcaffd700 1 -- 172.21.15.89:0/1397496046 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efdd0055d40 con 0x7efdd415c500 2023-07-06T02:22:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.843+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdd415dbe0 con 0x7efdd415c500 2023-07-06T02:22:58.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.843+0000 7efdcaffd700 1 -- 172.21.15.89:0/1397496046 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4670 (secure 0 0 0) 0x7efdd0059360 con 0x7efdd415c500 2023-07-06T02:22:58.845 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:58.846 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:22:58.459092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-06T02:22:58.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.845+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efdb806ace0 msgr2=0x7efdb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.845+0000 7efddc773700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efdb806ace0 0x7efdb806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006fd0 tx=0x7efdc4008040 comp rx=0 tx=0).stop 2023-07-06T02:22:58.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.845+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd415c500 msgr2=0x7efdd4159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.845+0000 7efddc773700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd415c500 0x7efdd4159320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdd4066d50 tx=0x7efdd0004750 comp rx=0 tx=0).stop 2023-07-06T02:22:58.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 shutdown_connections 2023-07-06T02:22:58.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd4160000 0x7efdd4158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd4070f20 0x7efdd40c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efdb806ace0 0x7efdb806d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 --2- 172.21.15.89:0/1397496046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdd415c500 0x7efdd4159320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 >> 172.21.15.89:0/1397496046 conn(0x7efdd40cf1c0 msgr2=0x7efdd415c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:58.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 shutdown_connections 2023-07-06T02:22:58.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.846+0000 7efddc773700 1 -- 172.21.15.89:0/1397496046 wait complete. 2023-07-06T02:22:58.850 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 6 2023-07-06T02:22:58.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:22:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.977+0000 7f6f1f2eb700 1 Processor -- start 2023-07-06T02:22:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.978+0000 7f6f1f2eb700 1 -- start start 2023-07-06T02:22:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.978+0000 7f6f1f2eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 0x7f6f180b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.978+0000 7f6f1f2eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f180cab50 con 0x7f6f1815c690 2023-07-06T02:22:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.978+0000 7f6f1f2eb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1815db70 con 0x7f6f180b1490 2023-07-06T02:22:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.978+0000 7f6f1f2eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1815de50 con 0x7f6f1806c6b0 2023-07-06T02:22:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6f1815c690 0x7f6f180cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56922/0 (socket says 172.21.15.89:56922) 2023-07-06T02:22:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 -- 172.21.15.89:0/269967556 learned_addr learned my addr 172.21.15.89:0/269967556 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:22:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d888700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 0x7f6f180b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574587197 0 0) 0x7f6f180cab50 con 0x7f6f1815c690 2023-07-06T02:22:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f040035b0 con 0x7f6f1815c690 2023-07-06T02:22:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888033302 0 0) 0x7f6f1815db70 con 0x7f6f180b1490 2023-07-06T02:22:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f180cab50 con 0x7f6f180b1490 2023-07-06T02:22:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463318701 0 0) 0x7f6f040035b0 con 0x7f6f1815c690 2023-07-06T02:22:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 msgr2=0x7f6f180b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 0x7f6f180b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 >> v1:172.21.15.172:6789/0 conn(0x7f6f180b1490 legacy=0x7f6f1806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180cad80 con 0x7f6f1815c690 2023-07-06T02:22:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f6f08001670 con 0x7f6f1815c690 2023-07-06T02:22:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 >> v1:172.21.15.89:6789/0 conn(0x7f6f1815c690 legacy=0x7f6f180cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f04003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f04004850 0x7f6f04004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 0x7f6f04005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1815de50 con 0x7f6f04003bc0 2023-07-06T02:22:58.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0400bac0 con 0x7f6f04004850 2023-07-06T02:22:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0400bc30 con 0x7f6f1806c6b0 2023-07-06T02:22:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f04003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1c886700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f04004850 0x7f6f04004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d888700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 0x7f6f04005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 -- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 msgr2=0x7f6f04005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f1806c6b0 0x7f6f04005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 -- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f04004850 msgr2=0x7f6f04004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f04004850 0x7f6f04004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1d087700 1 -- 172.21.15.89:0/269967556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f040035b0 con 0x7f6f04003bc0 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.979+0000 7f6f1c886700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f04004850 0x7f6f04004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1d087700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f04003fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c210 tx=0x7f6f04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb3aa93d0396bed server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800cdb0 con 0x7f6f04003bc0 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f08001af0 con 0x7f6f04003bc0 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f0ffff700 1 -- 172.21.15.89:0/269967556 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08005ba0 con 0x7f6f04003bc0 2023-07-06T02:22:58.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/269967556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 msgr2=0x7f6f04003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1f2eb700 1 --2- 172.21.15.89:0/269967556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f04003fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c210 tx=0x7f6f04005880 comp rx=0 tx=0).stop 2023-07-06T02:22:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1d888700 1 -- 172.21.15.89:0/269967556 reap_dead start 2023-07-06T02:22:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/269967556 shutdown_connections 2023-07-06T02:22:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/269967556 >> 172.21.15.89:0/269967556 conn(0x7f6f180cc490 msgr2=0x7f6f180cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/269967556 shutdown_connections 2023-07-06T02:22:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.980+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/269967556 wait complete. 2023-07-06T02:22:58.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 Processor -- start 2023-07-06T02:22:58.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 -- start start 2023-07-06T02:22:58.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f18159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f180ac390 0x7f6f1815d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1815a910 0x7f6f1815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180c3a50 con 0x7f6f04003bc0 2023-07-06T02:22:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1815b2d0 con 0x7f6f1815a910 2023-07-06T02:22:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1f2eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180ac900 con 0x7f6f180ac390 2023-07-06T02:22:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1d087700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f18159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1d087700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f18159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60264/0 (socket says 172.21.15.89:60264) 2023-07-06T02:22:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.982+0000 7f6f1d087700 1 -- 172.21.15.89:0/1336540216 learned_addr learned my addr 172.21.15.89:0/1336540216 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1c886700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f180ac390 0x7f6f1815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d888700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1815a910 0x7f6f1815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d087700 1 -- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f180ac390 msgr2=0x7f6f1815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d087700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f180ac390 0x7f6f1815d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d087700 1 -- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1815a910 msgr2=0x7f6f1815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:58.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d087700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1815a910 0x7f6f1815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d087700 1 -- 172.21.15.89:0/1336540216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180acbe0 con 0x7f6f04003bc0 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1c886700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f180ac390 0x7f6f1815d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d888700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1815a910 0x7f6f1815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1d087700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f18159c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c800 tx=0x7f6f08001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f0dffb700 1 -- 172.21.15.89:0/1336540216 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080072d0 con 0x7f6f04003bc0 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f0dffb700 1 -- 172.21.15.89:0/1336540216 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f08007430 con 0x7f6f04003bc0 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f180acec0 con 0x7f6f04003bc0 2023-07-06T02:22:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f0dffb700 1 -- 172.21.15.89:0/1336540216 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080077c0 con 0x7f6f04003bc0 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.983+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1815a140 con 0x7f6f04003bc0 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.984+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f180c37b0 con 0x7f6f04003bc0 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.984+0000 7f6f0dffb700 1 -- 172.21.15.89:0/1336540216 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f08027450 con 0x7f6f04003bc0 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.985+0000 7f6f0dffb700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6efc06ace0 0x7f6efc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.985+0000 7f6f0dffb700 1 -- 172.21.15.89:0/1336540216 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6f0808aef0 con 0x7f6f04003bc0 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.985+0000 7f6f1c886700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6efc06ace0 0x7f6efc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.985+0000 7f6f1c886700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6efc06ace0 0x7f6efc06d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f14001790 tx=0x7f6f14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:58.989+0000 7f6f0dffb700 1 -- 172.21.15.89:0/1336540216 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6f180c37b0 con 0x7f6f04003bc0 2023-07-06T02:22:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.178+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f180c37b0 con 0x7f6f04003bc0 2023-07-06T02:22:59.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.179+0000 7f6f0dffb700 1 -- 172.21.15.89:0/1336540216 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f6f180c37b0 con 0x7f6f04003bc0 2023-07-06T02:22:59.181 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:59.181 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:22:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6efc06ace0 msgr2=0x7f6efc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6efc06ace0 0x7f6efc06d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f14001790 tx=0x7f6f14009040 comp rx=0 tx=0).stop 2023-07-06T02:22:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 msgr2=0x7f6f18159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f18159c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c800 tx=0x7f6f08001980 comp rx=0 tx=0).stop 2023-07-06T02:22:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 shutdown_connections 2023-07-06T02:22:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1815a910 0x7f6f1815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6efc06ace0 0x7f6efc06d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f04003bc0 0x7f6f18159c00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 --2- 172.21.15.89:0/1336540216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f180ac390 0x7f6f1815d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 >> 172.21.15.89:0/1336540216 conn(0x7f6f180cc490 msgr2=0x7f6f1815b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 shutdown_connections 2023-07-06T02:22:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.181+0000 7f6f1f2eb700 1 -- 172.21.15.89:0/1336540216 wait complete. 2023-07-06T02:22:59.195 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:22:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.313+0000 7f1a55093700 1 Processor -- start 2023-07-06T02:22:59.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.313+0000 7f1a55093700 1 -- start start 2023-07-06T02:22:59.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a55093700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a50070f20 0x7f1a500713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a55093700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a500cb9f0 con 0x7f1a5015c500 2023-07-06T02:22:59.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a55093700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5015d8e0 con 0x7f1a50160000 2023-07-06T02:22:59.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a55093700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a50071900 con 0x7f1a50070f20 2023-07-06T02:22:59.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a50070f20 0x7f1a500713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a50070f20 0x7f1a500713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60274/0 (socket says 172.21.15.89:60274) 2023-07-06T02:22:59.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4e59c700 1 -- 172.21.15.89:0/45547968 learned_addr learned my addr 172.21.15.89:0/45547968 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:59.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4ed9d700 1 --1- 172.21.15.89:0/45547968 >> v1:172.21.15.89:6789/0 conn(0x7f1a5015c500 0x7f1a5006edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56942/0 (socket says 172.21.15.89:56942) 2023-07-06T02:22:59.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4e59c700 1 -- 172.21.15.89:0/45547968 >> v1:172.21.15.172:6789/0 conn(0x7f1a50160000 legacy=0x7f1a50160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4e59c700 1 -- 172.21.15.89:0/45547968 >> v1:172.21.15.89:6789/0 conn(0x7f1a5015c500 legacy=0x7f1a5006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4e59c700 1 -- 172.21.15.89:0/45547968 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50160be0 con 0x7f1a50070f20 2023-07-06T02:22:59.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4e59c700 1 --2- 172.21.15.89:0/45547968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50070f20 0x7f1a500713c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a440098f0 tx=0x7f1a44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b784e9e5101e94a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:59.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4dd9b700 1 -- 172.21.15.89:0/45547968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400b8a0 con 0x7f1a50070f20 2023-07-06T02:22:59.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4dd9b700 1 -- 172.21.15.89:0/45547968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4400ba00 con 0x7f1a50070f20 2023-07-06T02:22:59.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.314+0000 7f1a4dd9b700 1 -- 172.21.15.89:0/45547968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400bd90 con 0x7f1a50070f20 2023-07-06T02:22:59.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.315+0000 7f1a55093700 1 -- 172.21.15.89:0/45547968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50070f20 msgr2=0x7f1a500713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.315+0000 7f1a55093700 1 --2- 172.21.15.89:0/45547968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50070f20 0x7f1a500713c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a440098f0 tx=0x7f1a44009c00 comp rx=0 tx=0).stop 2023-07-06T02:22:59.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.315+0000 7f1a55093700 1 -- 172.21.15.89:0/45547968 shutdown_connections 2023-07-06T02:22:59.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.315+0000 7f1a55093700 1 --2- 172.21.15.89:0/45547968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50070f20 0x7f1a500713c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.315+0000 7f1a55093700 1 -- 172.21.15.89:0/45547968 >> 172.21.15.89:0/45547968 conn(0x7f1a500cf1c0 msgr2=0x7f1a500cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:59.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.315+0000 7f1a55093700 1 -- 172.21.15.89:0/45547968 shutdown_connections 2023-07-06T02:22:59.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.315+0000 7f1a55093700 1 -- 172.21.15.89:0/45547968 wait complete. 2023-07-06T02:22:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.316+0000 7f1a55093700 1 Processor -- start 2023-07-06T02:22:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.316+0000 7f1a55093700 1 -- start start 2023-07-06T02:22:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a55093700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a50070f20 0x7f1a500adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a55093700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a5015c500 0x7f1a5015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a55093700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50160000 0x7f1a5015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a55093700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500b3d20 con 0x7f1a50160000 2023-07-06T02:22:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a55093700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500b4ef0 con 0x7f1a50070f20 2023-07-06T02:22:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a55093700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500b33f0 con 0x7f1a5015c500 2023-07-06T02:22:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50160000 0x7f1a5015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50160000 0x7f1a5015ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60282/0 (socket says 172.21.15.89:60282) 2023-07-06T02:22:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4f59e700 1 -- 172.21.15.89:0/1078760774 learned_addr learned my addr 172.21.15.89:0/1078760774 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4ed9d700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a50070f20 0x7f1a500adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4e59c700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a5015c500 0x7f1a5015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4f59e700 1 -- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a5015c500 msgr2=0x7f1a5015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4f59e700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a5015c500 0x7f1a5015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4f59e700 1 -- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a50070f20 msgr2=0x7f1a500adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.317+0000 7f1a4f59e700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a50070f20 0x7f1a500adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.318+0000 7f1a4f59e700 1 -- 172.21.15.89:0/1078760774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44009550 con 0x7f1a50160000 2023-07-06T02:22:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.318+0000 7f1a4f59e700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50160000 0x7f1a5015ce10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800cb50 tx=0x7f1a4800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.318+0000 7f1a377fe700 1 -- 172.21.15.89:0/1078760774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48003c00 con 0x7f1a50160000 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.318+0000 7f1a377fe700 1 -- 172.21.15.89:0/1078760774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48003d60 con 0x7f1a50160000 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.318+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a500b36d0 con 0x7f1a50160000 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.318+0000 7f1a377fe700 1 -- 172.21.15.89:0/1078760774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800ab70 con 0x7f1a50160000 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.318+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a500b3050 con 0x7f1a50160000 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.319+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a500b3e90 con 0x7f1a50160000 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.319+0000 7f1a377fe700 1 -- 172.21.15.89:0/1078760774 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a4800acd0 con 0x7f1a50160000 2023-07-06T02:22:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.320+0000 7f1a377fe700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1a3c06ace0 0x7f1a3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.320+0000 7f1a4ed9d700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1a3c06ace0 0x7f1a3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.320+0000 7f1a377fe700 1 -- 172.21.15.89:0/1078760774 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1a48089810 con 0x7f1a50160000 2023-07-06T02:22:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.320+0000 7f1a4ed9d700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1a3c06ace0 0x7f1a3c06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a40000f60 tx=0x7f1a40007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.324+0000 7f1a377fe700 1 -- 172.21.15.89:0/1078760774 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a480545d0 con 0x7f1a50160000 2023-07-06T02:22:59.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.517+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a50074270 con 0x7f1a50160000 2023-07-06T02:22:59.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.518+0000 7f1a377fe700 1 -- 172.21.15.89:0/1078760774 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f1a48057bf0 con 0x7f1a50160000 2023-07-06T02:22:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:59.520 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:22:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1a3c06ace0 msgr2=0x7f1a3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1a3c06ace0 0x7f1a3c06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a40000f60 tx=0x7f1a40007d30 comp rx=0 tx=0).stop 2023-07-06T02:22:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50160000 msgr2=0x7f1a5015ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50160000 0x7f1a5015ce10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800cb50 tx=0x7f1a4800cf10 comp rx=0 tx=0).stop 2023-07-06T02:22:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 shutdown_connections 2023-07-06T02:22:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1a3c06ace0 0x7f1a3c06d190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a50160000 0x7f1a5015ce10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a50070f20 0x7f1a500adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 --2- 172.21.15.89:0/1078760774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a5015c500 0x7f1a5015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 >> 172.21.15.89:0/1078760774 conn(0x7f1a500cf1c0 msgr2=0x7f1a50160420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 shutdown_connections 2023-07-06T02:22:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.520+0000 7f1a55093700 1 -- 172.21.15.89:0/1078760774 wait complete. 2023-07-06T02:22:59.550 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:22:59.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.669+0000 7f0befb73700 1 Processor -- start 2023-07-06T02:22:59.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.669+0000 7f0befb73700 1 -- start start 2023-07-06T02:22:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0befb73700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0be8070f20 0x7f0be80c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0befb73700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0be80712f0 0x7f0be806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0befb73700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be806e790 0x7f0be806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0befb73700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be80c9fa0 con 0x7f0be80712f0 2023-07-06T02:22:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0befb73700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be806f290 con 0x7f0be8070f20 2023-07-06T02:22:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0befb73700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be806ecd0 con 0x7f0be806e790 2023-07-06T02:22:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0be80712f0 0x7f0be806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0be80712f0 0x7f0be806e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60286/0 (socket says 172.21.15.89:60286) 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 -- 172.21.15.89:0/1928267223 learned_addr learned my addr 172.21.15.89:0/1928267223 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed90f700 1 --2- 172.21.15.89:0/1928267223 >> v2:172.21.15.172:3300/0 conn(0x7f0be8070f20 0x7f0be80c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bee110700 1 --2- 172.21.15.89:0/1928267223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be806e790 0x7f0be806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 -- 172.21.15.89:0/1928267223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be806e790 msgr2=0x7f0be806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 --2- 172.21.15.89:0/1928267223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be806e790 0x7f0be806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 -- 172.21.15.89:0/1928267223 >> v2:172.21.15.172:3300/0 conn(0x7f0be8070f20 msgr2=0x7f0be80c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:22:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 --2- 172.21.15.89:0/1928267223 >> v2:172.21.15.172:3300/0 conn(0x7f0be8070f20 0x7f0be80c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bed10e700 1 -- 172.21.15.89:0/1928267223 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be806efb0 con 0x7f0be80712f0 2023-07-06T02:22:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.670+0000 7f0bee110700 1 --2- 172.21.15.89:0/1928267223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be806e790 0x7f0be806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0bed90f700 1 --2- 172.21.15.89:0/1928267223 >> v2:172.21.15.172:3300/0 conn(0x7f0be8070f20 0x7f0be80c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:22:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0bed10e700 1 --2- 172.21.15.89:0/1928267223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be80712f0 0x7f0be806e250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0be806cd40 tx=0x7f0be4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49dab17c42437ee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0bec90d700 1 -- 172.21.15.89:0/1928267223 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400b870 con 0x7f0be80712f0 2023-07-06T02:22:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0bec90d700 1 -- 172.21.15.89:0/1928267223 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be400b9d0 con 0x7f0be80712f0 2023-07-06T02:22:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0bec90d700 1 -- 172.21.15.89:0/1928267223 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400bd60 con 0x7f0be80712f0 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 -- 172.21.15.89:0/1928267223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be80712f0 msgr2=0x7f0be806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 --2- 172.21.15.89:0/1928267223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be80712f0 0x7f0be806e250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0be806cd40 tx=0x7f0be4009bd0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 -- 172.21.15.89:0/1928267223 shutdown_connections 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 --2- 172.21.15.89:0/1928267223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be806e790 0x7f0be806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 --2- 172.21.15.89:0/1928267223 >> v2:172.21.15.172:3300/0 conn(0x7f0be8070f20 0x7f0be80c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 --2- 172.21.15.89:0/1928267223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be80712f0 0x7f0be806e250 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 -- 172.21.15.89:0/1928267223 >> 172.21.15.89:0/1928267223 conn(0x7f0be80cf1c0 msgr2=0x7f0be80cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 -- 172.21.15.89:0/1928267223 shutdown_connections 2023-07-06T02:22:59.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.671+0000 7f0befb73700 1 -- 172.21.15.89:0/1928267223 wait complete. 2023-07-06T02:22:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 Processor -- start 2023-07-06T02:22:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 -- start start 2023-07-06T02:22:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be806e790 0x7f0be81499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be8070f20 0x7f0be80afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be80712f0 0x7f0be8149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80afa10 con 0x7f0be806e790 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be814ca90 con 0x7f0be8070f20 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0befb73700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be814cd70 con 0x7f0be80712f0 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0bed90f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be806e790 0x7f0be81499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0bed90f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be806e790 0x7f0be81499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60292/0 (socket says 172.21.15.89:60292) 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.673+0000 7f0bed90f700 1 -- 172.21.15.89:0/2102294483 learned_addr learned my addr 172.21.15.89:0/2102294483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:22:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bed10e700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be8070f20 0x7f0be80afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bee110700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be80712f0 0x7f0be8149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bed90f700 1 -- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be80712f0 msgr2=0x7f0be8149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bed90f700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be80712f0 0x7f0be8149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bed90f700 1 -- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be8070f20 msgr2=0x7f0be80afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bed90f700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be8070f20 0x7f0be80afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bed90f700 1 -- 172.21.15.89:0/2102294483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4009550 con 0x7f0be806e790 2023-07-06T02:22:59.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bed90f700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be806e790 0x7f0be81499b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c750 tx=0x7f0bd800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bde7fc700 1 -- 172.21.15.89:0/2102294483 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8011800 con 0x7f0be806e790 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bde7fc700 1 -- 172.21.15.89:0/2102294483 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd8011960 con 0x7f0be806e790 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be80af6a0 con 0x7f0be806e790 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0bde7fc700 1 -- 172.21.15.89:0/2102294483 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8011cd0 con 0x7f0be806e790 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.674+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be80c2ed0 con 0x7f0be806e790 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.675+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be80afcf0 con 0x7f0be806e790 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.675+0000 7f0bde7fc700 1 -- 172.21.15.89:0/2102294483 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0bd8021350 con 0x7f0be806e790 2023-07-06T02:22:59.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.676+0000 7f0bde7fc700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0bcc06ace0 0x7f0bcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:22:59.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.676+0000 7f0bed10e700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0bcc06ace0 0x7f0bcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:22:59.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.676+0000 7f0bde7fc700 1 -- 172.21.15.89:0/2102294483 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0bd8089780 con 0x7f0be806e790 2023-07-06T02:22:59.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.676+0000 7f0bed10e700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0bcc06ace0 0x7f0bcc06d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009840 tx=0x7f0be4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:22:59.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.680+0000 7f0bde7fc700 1 -- 172.21.15.89:0/2102294483 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0bd8054530 con 0x7f0be806e790 2023-07-06T02:22:59.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.908+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0be8066fb0 con 0x7f0be806e790 2023-07-06T02:22:59.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.908+0000 7f0bde7fc700 1 -- 172.21.15.89:0/2102294483 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f0bd8057b50 con 0x7f0be806e790 2023-07-06T02:22:59.910 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:22:59.911 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:22:58.459092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-06T02:22:59.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0bcc06ace0 msgr2=0x7f0bcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0bcc06ace0 0x7f0bcc06d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009840 tx=0x7f0be4008030 comp rx=0 tx=0).stop 2023-07-06T02:22:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be806e790 msgr2=0x7f0be81499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:22:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be806e790 0x7f0be81499b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c750 tx=0x7f0bd800cb10 comp rx=0 tx=0).stop 2023-07-06T02:22:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 shutdown_connections 2023-07-06T02:22:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0be80712f0 0x7f0be8149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0bcc06ace0 0x7f0bcc06d190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0be806e790 0x7f0be81499b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 --2- 172.21.15.89:0/2102294483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be8070f20 0x7f0be80afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:22:59.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.910+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 >> 172.21.15.89:0/2102294483 conn(0x7f0be80cf1c0 msgr2=0x7f0be815c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:22:59.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.911+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 shutdown_connections 2023-07-06T02:22:59.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:22:59.911+0000 7f0befb73700 1 -- 172.21.15.89:0/2102294483 wait complete. 2023-07-06T02:22:59.914 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 6 2023-07-06T02:22:59.924 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.043+0000 7ff807fb4700 1 Processor -- start 2023-07-06T02:23:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.043+0000 7ff807fb4700 1 -- start start 2023-07-06T02:23:00.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff807fb4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80015a3e0 0x7ff8000caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff807fb4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff800071630 0x7ff80015dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff807fb4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80006a500 con 0x7ff80006bcd0 2023-07-06T02:23:00.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff807fb4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80006ec60 con 0x7ff800071630 2023-07-06T02:23:00.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff807fb4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80006f190 con 0x7ff80015a3e0 2023-07-06T02:23:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff806551700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff800071630 0x7ff80015dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff80554f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff80006bcd0 0x7ff800070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56970/0 (socket says 172.21.15.89:56970) 2023-07-06T02:23:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff80554f700 1 -- 172.21.15.89:0/1376559027 learned_addr learned my addr 172.21.15.89:0/1376559027 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff805d50700 1 --2- 172.21.15.89:0/1376559027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80015a3e0 0x7ff8000caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff806551700 1 -- 172.21.15.89:0/1376559027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80015a3e0 msgr2=0x7ff8000caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff806551700 1 --2- 172.21.15.89:0/1376559027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80015a3e0 0x7ff8000caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff806551700 1 -- 172.21.15.89:0/1376559027 >> v1:172.21.15.89:6789/0 conn(0x7ff80006bcd0 legacy=0x7ff800070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff804d4e700 1 -- 172.21.15.89:0/1376559027 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112019996 0 0) 0x7ff80006a500 con 0x7ff80006bcd0 2023-07-06T02:23:00.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff806551700 1 -- 172.21.15.89:0/1376559027 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000cb980 con 0x7ff800071630 2023-07-06T02:23:00.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.044+0000 7ff805d50700 1 --2- 172.21.15.89:0/1376559027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80015a3e0 0x7ff8000caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:00.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff806551700 1 --2- 172.21.15.89:0/1376559027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff80015dda0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009d90 tx=0x7ff7f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9849d4194ce2c3a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff804d4e700 1 -- 172.21.15.89:0/1376559027 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800e070 con 0x7ff800071630 2023-07-06T02:23:00.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff804d4e700 1 -- 172.21.15.89:0/1376559027 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f800bbd0 con 0x7ff800071630 2023-07-06T02:23:00.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff804d4e700 1 -- 172.21.15.89:0/1376559027 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8012430 con 0x7ff800071630 2023-07-06T02:23:00.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 -- 172.21.15.89:0/1376559027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 msgr2=0x7ff80015dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 --2- 172.21.15.89:0/1376559027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff80015dda0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009d90 tx=0x7ff7f800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:00.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 -- 172.21.15.89:0/1376559027 shutdown_connections 2023-07-06T02:23:00.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 --2- 172.21.15.89:0/1376559027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff80015dda0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 --2- 172.21.15.89:0/1376559027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80015a3e0 0x7ff8000caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 -- 172.21.15.89:0/1376559027 >> 172.21.15.89:0/1376559027 conn(0x7ff8000cf1c0 msgr2=0x7ff8000cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 -- 172.21.15.89:0/1376559027 shutdown_connections 2023-07-06T02:23:00.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.045+0000 7ff807fb4700 1 -- 172.21.15.89:0/1376559027 wait complete. 2023-07-06T02:23:00.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.046+0000 7ff807fb4700 1 Processor -- start 2023-07-06T02:23:00.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff807fb4700 1 -- start start 2023-07-06T02:23:00.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff807fb4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80006bcd0 0x7ff8000b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff807fb4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff8001557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff807fb4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80015a3e0 0x7ff8000ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff807fb4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000cd5e0 con 0x7ff800071630 2023-07-06T02:23:00.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff807fb4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000cd8c0 con 0x7ff80015a3e0 2023-07-06T02:23:00.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff807fb4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000cdba0 con 0x7ff80006bcd0 2023-07-06T02:23:00.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff80554f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff8001557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff80554f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff8001557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60308/0 (socket says 172.21.15.89:60308) 2023-07-06T02:23:00.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.047+0000 7ff80554f700 1 -- 172.21.15.89:0/2778730572 learned_addr learned my addr 172.21.15.89:0/2778730572 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff805d50700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80006bcd0 0x7ff8000b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff806551700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80015a3e0 0x7ff8000ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff80554f700 1 -- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80006bcd0 msgr2=0x7ff8000b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff80554f700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80006bcd0 0x7ff8000b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff80554f700 1 -- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80015a3e0 msgr2=0x7ff8000ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff80554f700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80015a3e0 0x7ff8000ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff80554f700 1 -- 172.21.15.89:0/2778730572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f8009a70 con 0x7ff800071630 2023-07-06T02:23:00.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff805d50700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80006bcd0 0x7ff8000b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:00.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff80554f700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff8001557c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c550 tx=0x7ff7fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff7f67fc700 1 -- 172.21.15.89:0/2778730572 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc004c80 con 0x7ff800071630 2023-07-06T02:23:00.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff7f67fc700 1 -- 172.21.15.89:0/2778730572 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc004e00 con 0x7ff800071630 2023-07-06T02:23:00.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8000cde80 con 0x7ff800071630 2023-07-06T02:23:00.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff7f67fc700 1 -- 172.21.15.89:0/2778730572 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007310 con 0x7ff800071630 2023-07-06T02:23:00.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.048+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff800155d00 con 0x7ff800071630 2023-07-06T02:23:00.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.049+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8000ce3a0 con 0x7ff800071630 2023-07-06T02:23:00.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.049+0000 7ff7f67fc700 1 -- 172.21.15.89:0/2778730572 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7fc007470 con 0x7ff800071630 2023-07-06T02:23:00.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.050+0000 7ff7f67fc700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7e406ace0 0x7ff7e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.050+0000 7ff805d50700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7e406ace0 0x7ff7e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.050+0000 7ff7f67fc700 1 -- 172.21.15.89:0/2778730572 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff7fc0893f0 con 0x7ff800071630 2023-07-06T02:23:00.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.050+0000 7ff805d50700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7e406ace0 0x7ff7e406d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009a50 tx=0x7ff7f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.054+0000 7ff7f67fc700 1 -- 172.21.15.89:0/2778730572 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff8000ce3a0 con 0x7ff800071630 2023-07-06T02:23:00.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.246+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8000ce3a0 con 0x7ff800071630 2023-07-06T02:23:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.247+0000 7ff7f67fc700 1 -- 172.21.15.89:0/2778730572 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7ff8000ce3a0 con 0x7ff800071630 2023-07-06T02:23:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:00.250 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7e406ace0 msgr2=0x7ff7e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7e406ace0 0x7ff7e406d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009a50 tx=0x7ff7f0008040 comp rx=0 tx=0).stop 2023-07-06T02:23:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 msgr2=0x7ff8001557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff8001557c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c550 tx=0x7ff7fc00c910 comp rx=0 tx=0).stop 2023-07-06T02:23:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 shutdown_connections 2023-07-06T02:23:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80015a3e0 0x7ff8000ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff80006bcd0 0x7ff8000b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7e406ace0 0x7ff7e406d190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 --2- 172.21.15.89:0/2778730572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800071630 0x7ff8001557c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 >> 172.21.15.89:0/2778730572 conn(0x7ff8000cf1c0 msgr2=0x7ff8000b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 shutdown_connections 2023-07-06T02:23:00.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.250+0000 7ff807fb4700 1 -- 172.21.15.89:0/2778730572 wait complete. 2023-07-06T02:23:00.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T02:22:52.797894+0000', 'modified': '2023-07-06T02:22:58.459092+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/1524050092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 1524050092}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 1524050092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-06T02:23:00.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T02:23:00.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T02:23:00.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.787+0000 7f65eb497700 1 Processor -- start 2023-07-06T02:23:00.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65eb497700 1 -- start start 2023-07-06T02:23:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65eb497700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65e4156ab0 0x7f65e4160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65eb497700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e4160bb0 0x7f65e40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65eb497700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40cb3d0 con 0x7f65e41600b0 2023-07-06T02:23:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65eb497700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e4156e80 con 0x7f65e4156ab0 2023-07-06T02:23:00.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65eb497700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40ca370 con 0x7f65e4160bb0 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65e9233700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65e4156ab0 0x7f65e4160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.788+0000 7f65e9233700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f65e4156ab0 0x7f65e4160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60314/0 (socket says 172.21.15.89:60314) 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65e9233700 1 -- 172.21.15.89:0/2861148988 learned_addr learned my addr 172.21.15.89:0/2861148988 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65e9233700 1 -- 172.21.15.89:0/2861148988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e4160bb0 msgr2=0x7f65e40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65e9233700 1 --2- 172.21.15.89:0/2861148988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e4160bb0 0x7f65e40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65e9233700 1 -- 172.21.15.89:0/2861148988 >> v1:172.21.15.172:6789/0 conn(0x7f65e41600b0 legacy=0x7f65e406d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65e9233700 1 -- 172.21.15.89:0/2861148988 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40ca610 con 0x7f65e4156ab0 2023-07-06T02:23:00.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65e9233700 1 --2- 172.21.15.89:0/2861148988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4160670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098f0 tx=0x7f65d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5c1ce435a869c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65dbfff700 1 -- 172.21.15.89:0/2861148988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4004230 con 0x7f65e4156ab0 2023-07-06T02:23:00.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65dbfff700 1 -- 172.21.15.89:0/2861148988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4004390 con 0x7f65e4156ab0 2023-07-06T02:23:00.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.789+0000 7f65dbfff700 1 -- 172.21.15.89:0/2861148988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40036a0 con 0x7f65e4156ab0 2023-07-06T02:23:00.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 -- 172.21.15.89:0/2861148988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 msgr2=0x7f65e4160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 --2- 172.21.15.89:0/2861148988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4160670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098f0 tx=0x7f65d4009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 -- 172.21.15.89:0/2861148988 shutdown_connections 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 --2- 172.21.15.89:0/2861148988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4160670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 --2- 172.21.15.89:0/2861148988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e4160bb0 0x7f65e40b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 -- 172.21.15.89:0/2861148988 >> 172.21.15.89:0/2861148988 conn(0x7f65e40cc490 msgr2=0x7f65e40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 -- 172.21.15.89:0/2861148988 shutdown_connections 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.790+0000 7f65eb497700 1 -- 172.21.15.89:0/2861148988 wait complete. 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.791+0000 7f65eb497700 1 Processor -- start 2023-07-06T02:23:00.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.791+0000 7f65eb497700 1 -- start start 2023-07-06T02:23:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65eb497700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65eb497700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e41600b0 0x7f65e40b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65eb497700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e406b1e0 0x7f65e41f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65eb497700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e41e67a0 con 0x7f65e4156ab0 2023-07-06T02:23:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65eb497700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4155ed0 con 0x7f65e41600b0 2023-07-06T02:23:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65eb497700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40b2eb0 con 0x7f65e406b1e0 2023-07-06T02:23:00.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60320/0 (socket says 172.21.15.89:60320) 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.792+0000 7f65e9233700 1 -- 172.21.15.89:0/842031195 learned_addr learned my addr 172.21.15.89:0/842031195 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65e9a34700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e406b1e0 0x7f65e41f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65e9233700 1 -- 172.21.15.89:0/842031195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e406b1e0 msgr2=0x7f65e41f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65e9233700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e406b1e0 0x7f65e41f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65e9233700 1 -- 172.21.15.89:0/842031195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e41600b0 msgr2=0x7f65e40b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65e9233700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e41600b0 0x7f65e40b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65e9233700 1 -- 172.21.15.89:0/842031195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009550 con 0x7f65e4156ab0 2023-07-06T02:23:00.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65e9233700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4156110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009130 tx=0x7f65d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65d9ffb700 1 -- 172.21.15.89:0/842031195 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4003e70 con 0x7f65e4156ab0 2023-07-06T02:23:00.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65d9ffb700 1 -- 172.21.15.89:0/842031195 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d401f070 con 0x7f65e4156ab0 2023-07-06T02:23:00.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e41e5e70 con 0x7f65e4156ab0 2023-07-06T02:23:00.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65d9ffb700 1 -- 172.21.15.89:0/842031195 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400b8a0 con 0x7f65e4156ab0 2023-07-06T02:23:00.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.793+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e41e6430 con 0x7f65e4156ab0 2023-07-06T02:23:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.797+0000 7f65d9ffb700 1 -- 172.21.15.89:0/842031195 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65d4021030 con 0x7f65e4156ab0 2023-07-06T02:23:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.797+0000 7f65f761cb80 1 Processor -- start 2023-07-06T02:23:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.797+0000 7f65d9ffb700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65c806ad10 0x7f65c806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.797+0000 7f65f761cb80 1 -- start start 2023-07-06T02:23:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.797+0000 7f65d9ffb700 1 -- 172.21.15.89:0/842031195 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65d408aef0 con 0x7f65e4156ab0 2023-07-06T02:23:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.797+0000 7f65e9a34700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65c806ad10 0x7f65c806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65c806ad10 0x7f65c806d1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65dc001ce0 tx=0x7f65dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059ce470 0x55d8059ed670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059a4830 0x55d8059d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059cecd0 0x55d8059d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65f761cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8058abc60 con 0x55d8059cecd0 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8058fe9f0 con 0x55d8059a4830 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8058f7e80 con 0x55d8059ce470 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059cecd0 0x55d8059d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059cecd0 0x55d8059d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60332/0 (socket says 172.21.15.89:60332) 2023-07-06T02:23:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 -- 172.21.15.89:0/2912886515 learned_addr learned my addr 172.21.15.89:0/2912886515 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 -- 172.21.15.89:0/2912886515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059ce470 msgr2=0x55d8059ed670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e8a32700 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059a4830 0x55d8059d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059ce470 0x55d8059ed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 -- 172.21.15.89:0/2912886515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059a4830 msgr2=0x55d8059d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059a4830 0x55d8059d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 -- 172.21.15.89:0/2912886515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d805725d90 con 0x55d8059cecd0 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65e9a34700 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059cecd0 0x55d8059d1730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65dc003e30 tx=0x7f65dc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11ae415d480a7c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.798+0000 7f65ceffd700 1 -- 172.21.15.89:0/2912886515 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00da90 con 0x55d8059cecd0 2023-07-06T02:23:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65ceffd700 1 -- 172.21.15.89:0/2912886515 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc00dc10 con 0x55d8059cecd0 2023-07-06T02:23:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65ceffd700 1 -- 172.21.15.89:0/2912886515 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc003e60 con 0x55d8059cecd0 2023-07-06T02:23:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 -- 172.21.15.89:0/2912886515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059cecd0 msgr2=0x55d8059d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059cecd0 0x55d8059d1730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65dc003e30 tx=0x7f65dc010000 comp rx=0 tx=0).stop 2023-07-06T02:23:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 -- 172.21.15.89:0/2912886515 shutdown_connections 2023-07-06T02:23:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059cecd0 0x55d8059d1730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059ce470 0x55d8059ed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2912886515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059a4830 0x55d8059d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 -- 172.21.15.89:0/2912886515 >> 172.21.15.89:0/2912886515 conn(0x55d80599be80 msgr2=0x55d8059ec1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 -- 172.21.15.89:0/2912886515 shutdown_connections 2023-07-06T02:23:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.799+0000 7f65f761cb80 1 -- 172.21.15.89:0/2912886515 wait complete. 2023-07-06T02:23:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.800+0000 7f65f761cb80 1 -- start start 2023-07-06T02:23:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.800+0000 7f65f761cb80 1 Processor -- start 2023-07-06T02:23:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d805897a70 con 0x55d8059a4830 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d805897770 con 0x55d8059ce470 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d80577b8d0 con 0x55d8059cecd0 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 10 client.0 _ll_register_callbacks cb 0x55d805895710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059e6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60334/0 (socket says 172.21.15.89:60334) 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 -- 172.21.15.89:0/2995536186 learned_addr learned my addr 172.21.15.89:0/2995536186 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e8a32700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9a34700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 -- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 msgr2=0x55d8059cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 -- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 msgr2=0x55d8059fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 -- 172.21.15.89:0/2995536186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc003b40 con 0x55d8059a4830 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e8a32700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9a34700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65e9233700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059e6ba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65d4055e70 tx=0x7f65d405d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1648c179ac09b3c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65c3fff700 1 -- 172.21.15.89:0/2995536186 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4071070 con 0x55d8059a4830 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65c3fff700 1 -- 172.21.15.89:0/2995536186 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65d4057dd0 con 0x55d8059a4830 2023-07-06T02:23:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d805999490 con 0x55d8059a4830 2023-07-06T02:23:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65c3fff700 1 -- 172.21.15.89:0/2995536186 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65d4059060 con 0x55d8059a4830 2023-07-06T02:23:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 10 client.4452 fetch_fsmap learned FSMap version 6 2023-07-06T02:23:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.801+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d80570ec10 con 0x55d8059a4830 2023-07-06T02:23:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65c3fff700 1 -- 172.21.15.89:0/2995536186 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d805999490 con 0x55d8059a4830 2023-07-06T02:23:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65c3fff700 1 -- 172.21.15.89:0/2995536186 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f65d4078250 con 0x55d8059a4830 2023-07-06T02:23:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65f761cb80 10 client.4452 fetch_fsmap finished waiting for FSMap version 6 2023-07-06T02:23:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65f761cb80 10 client.4452 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65f761cb80 20 client.4452 populate_metadata read hostname 'smithi089' 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059ff860 0x55d8059e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65f761cb80 4 client.4452 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 --> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8058f0e60 con 0x55d8059ff860 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65e8a32700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059ff860 0x55d8059e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65c3fff700 10 client.4452 ms_handle_connect on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.802+0000 7f65e8a32700 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059ff860 0x55d8059e57b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd03b5d20920da5e server_cookie=9b5dd43decbd7ed5 in_seq=0 out_seq=0 2023-07-06T02:23:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65c3fff700 1 -- 172.21.15.89:0/2995536186 <== mds.0 v2:172.21.15.89:6836/1524050092 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d8058f0e60 con 0x55d8059ff860 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65c3fff700 10 client.4452 handle_command_reply: tid=0 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 client.4452 shutdown 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 msgr2=0x55d8059e6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059e6ba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65d4055e70 tx=0x7f65d405d930 comp rx=0 tx=0).stop 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 shutdown_connections 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059ff860 0x55d8059e57b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65c3fff700 0 client.4452 ms_handle_reset on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059e6ba0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 --2- 172.21.15.89:0/2995536186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 >> 172.21.15.89:0/2995536186 conn(0x55d80599be80 msgr2=0x55d8059a15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 shutdown_connections 2023-07-06T02:23:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 1 -- 172.21.15.89:0/2995536186 wait complete. 2023-07-06T02:23:00.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.803+0000 7f65f761cb80 20 client.4452 trim_cache size 0 max 16384 2023-07-06T02:23:00.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 Processor -- start 2023-07-06T02:23:00.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 -- start start 2023-07-06T02:23:00.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80599d650 con 0x55d8059a4830 2023-07-06T02:23:00.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8059feac0 con 0x55d8059ce470 2023-07-06T02:23:00.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8059ef110 con 0x55d8059cecd0 2023-07-06T02:23:00.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059a6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60342/0 (socket says 172.21.15.89:60342) 2023-07-06T02:23:00.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.821+0000 7f65e9233700 1 -- 172.21.15.89:0/3559996252 learned_addr learned my addr 172.21.15.89:0/3559996252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65e9233700 1 -- 172.21.15.89:0/3559996252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 msgr2=0x55d8059e30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65e9233700 1 --2- 172.21.15.89:0/3559996252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65e9233700 1 -- 172.21.15.89:0/3559996252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 msgr2=0x55d8059e2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65e9233700 1 --2- 172.21.15.89:0/3559996252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65e9233700 1 -- 172.21.15.89:0/3559996252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d80574c200 con 0x55d8059a4830 2023-07-06T02:23:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65e9233700 1 --2- 172.21.15.89:0/3559996252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059a6bc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65d4000c00 tx=0x7f65d4025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d159947fdae5d43b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65cf7fe700 1 -- 172.21.15.89:0/3559996252 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4071040 con 0x55d8059a4830 2023-07-06T02:23:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65cf7fe700 1 -- 172.21.15.89:0/3559996252 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4057dd0 con 0x55d8059a4830 2023-07-06T02:23:00.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65cf7fe700 1 -- 172.21.15.89:0/3559996252 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40590a0 con 0x55d8059a4830 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65f761cb80 1 -- 172.21.15.89:0/3559996252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 msgr2=0x55d8059a6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3559996252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059a6bc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65d4000c00 tx=0x7f65d4025000 comp rx=0 tx=0).stop 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65f761cb80 1 -- 172.21.15.89:0/3559996252 shutdown_connections 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3559996252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3559996252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d8059a6bc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3559996252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059e2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.822+0000 7f65f761cb80 1 -- 172.21.15.89:0/3559996252 >> 172.21.15.89:0/3559996252 conn(0x55d80599be80 msgr2=0x55d8059cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.823+0000 7f65f761cb80 1 -- 172.21.15.89:0/3559996252 shutdown_connections 2023-07-06T02:23:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.823+0000 7f65f761cb80 1 -- 172.21.15.89:0/3559996252 wait complete. 2023-07-06T02:23:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.823+0000 7f65f761cb80 1 -- start start 2023-07-06T02:23:00.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.823+0000 7f65f761cb80 1 Processor -- start 2023-07-06T02:23:00.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.824+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d805a06060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.824+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8059fb750 con 0x55d8059a4830 2023-07-06T02:23:00.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8059e2ec0 con 0x55d8059ce470 2023-07-06T02:23:00.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8059cc370 con 0x55d8059cecd0 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d805a06060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 10 client.0 _ll_register_callbacks cb 0x7f65d4059560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d805a06060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60348/0 (socket says 172.21.15.89:60348) 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 -- 172.21.15.89:0/3209113800 learned_addr learned my addr 172.21.15.89:0/3209113800 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e8a32700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9a34700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 -- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 msgr2=0x55d8059a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 -- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 msgr2=0x55d8059a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 -- 172.21.15.89:0/3209113800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e41e6430 con 0x55d8059a4830 2023-07-06T02:23:00.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e8a32700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:00.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65e9233700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d805a06060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009c40 tx=0x7f65d405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b431d356958f2087 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-07-06T02:23:00.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65c3fff700 1 -- 172.21.15.89:0/3209113800 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4071040 con 0x55d8059a4830 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65c3fff700 1 -- 172.21.15.89:0/3209113800 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65d4061830 con 0x55d8059a4830 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65dc00dc10 con 0x55d8059a4830 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65c3fff700 1 -- 172.21.15.89:0/3209113800 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65d40619f0 con 0x55d8059a4830 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 10 client.4458 fetch_fsmap learned FSMap version 6 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d805724260 con 0x55d8059a4830 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65c3fff700 1 -- 172.21.15.89:0/3209113800 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d407d020 con 0x55d8059a4830 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65c3fff700 1 -- 172.21.15.89:0/3209113800 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f65d4055760 con 0x55d8059a4830 2023-07-06T02:23:00.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 10 client.4458 fetch_fsmap finished waiting for FSMap version 6 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 10 client.4458 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 20 client.4458 populate_metadata read hostname 'smithi089' 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.825+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059a0f20 0x55d805a55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 4 client.4458 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 --> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d8058b3000 con 0x55d8059a0f20 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65e8a32700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059a0f20 0x55d805a55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65c3fff700 10 client.4458 ms_handle_connect on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65e8a32700 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059a0f20 0x55d805a55b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828c470204e4de08 server_cookie=bd5cbea4b2605a20 in_seq=0 out_seq=0 2023-07-06T02:23:00.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65c3fff700 1 -- 172.21.15.89:0/3209113800 <== mds.0 v2:172.21.15.89:6836/1524050092 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55d8058b3000 con 0x55d8059a0f20 2023-07-06T02:23:00.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65c3fff700 10 client.4458 handle_command_reply: tid=0 2023-07-06T02:23:00.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 client.4458 shutdown 2023-07-06T02:23:00.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 msgr2=0x55d805a06060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d805a06060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009c40 tx=0x7f65d405b000 comp rx=0 tx=0).stop 2023-07-06T02:23:00.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 shutdown_connections 2023-07-06T02:23:00.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x55d8059a0f20 0x55d805a55b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d8059cecd0 0x55d8059a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8059a4830 0x55d805a06060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65c3fff700 0 client.4458 ms_handle_reset on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 --2- 172.21.15.89:0/3209113800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8059ce470 0x55d8059a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 >> 172.21.15.89:0/3209113800 conn(0x55d80599be80 msgr2=0x55d8059ebb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 shutdown_connections 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.826+0000 7f65f761cb80 1 -- 172.21.15.89:0/3209113800 wait complete. 2023-07-06T02:23:00.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65f761cb80 20 client.4458 trim_cache size 0 max 16384 2023-07-06T02:23:00.839 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-07-06T02:23:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "b61ed025-dcde-49f5-a87d-8904066cfa54", 2023-07-06T02:23:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-07-06T02:23:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4299, 2023-07-06T02:23:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 7.4249155409999998, 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 6, 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 24, 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 24, 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 13.38485184 2023-07-06T02:23:00.840 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-07-06T02:23:00.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65c806ad10 msgr2=0x7f65c806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65c806ad10 0x7f65c806d1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65dc001ce0 tx=0x7f65dc009040 comp rx=0 tx=0).stop 2023-07-06T02:23:00.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 msgr2=0x7f65e4156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4156110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009130 tx=0x7f65d4004750 comp rx=0 tx=0).stop 2023-07-06T02:23:00.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 shutdown_connections 2023-07-06T02:23:00.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65e406b1e0 0x7f65e41f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65c806ad10 0x7f65c806d1c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e4156ab0 0x7f65e4156110 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 --2- 172.21.15.89:0/842031195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e41600b0 0x7f65e40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 >> 172.21.15.89:0/842031195 conn(0x7f65e40cc490 msgr2=0x7f65e415f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 shutdown_connections 2023-07-06T02:23:00.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.827+0000 7f65eb497700 1 -- 172.21.15.89:0/842031195 wait complete. 2023-07-06T02:23:00.843 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T02:23:00.844 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:00.845 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T02:23:00.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.962+0000 7f4159363700 1 Processor -- start 2023-07-06T02:23:00.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.962+0000 7f4159363700 1 -- start start 2023-07-06T02:23:00.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4159363700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f415406bdb0 0x7f415406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4159363700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 0x7f415415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4159363700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41540cd240 con 0x7f415406b610 2023-07-06T02:23:00.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4159363700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41540cf280 con 0x7f415406bdb0 2023-07-06T02:23:00.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4159363700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cd490 con 0x7f41540ceda0 2023-07-06T02:23:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f41527fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f415406b610 0x7f415406c180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57028/0 (socket says 172.21.15.89:57028) 2023-07-06T02:23:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f41527fc700 1 -- 172.21.15.89:0/1772315828 learned_addr learned my addr 172.21.15.89:0/1772315828 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4152ffd700 1 --2- 172.21.15.89:0/1772315828 >> v2:172.21.15.172:3300/0 conn(0x7f415406bdb0 0x7f415406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f41537fe700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 0x7f415415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162463280 0 0) 0x7f41540cd240 con 0x7f415406b610 2023-07-06T02:23:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f413c0035b0 con 0x7f415406b610 2023-07-06T02:23:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2880483973 0 0) 0x7f413c0035b0 con 0x7f415406b610 2023-07-06T02:23:00.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 msgr2=0x7f415415c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 0x7f415415c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 >> v2:172.21.15.172:3300/0 conn(0x7f415406bdb0 msgr2=0x7f415406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 --2- 172.21.15.89:0/1772315828 >> v2:172.21.15.172:3300/0 conn(0x7f415406bdb0 0x7f415406aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41540cd770 con 0x7f415406b610 2023-07-06T02:23:00.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f4148001670 con 0x7f415406b610 2023-07-06T02:23:00.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 >> v1:172.21.15.89:6789/0 conn(0x7f415406b610 legacy=0x7f415406c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f413c003a00 0x7f413c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 0x7f413c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f413c004d90 0x7f413c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cf280 con 0x7f413c004d90 2023-07-06T02:23:00.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cd490 con 0x7f413c003a00 2023-07-06T02:23:00.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.963+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c00ba10 con 0x7f41540ceda0 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41537fe700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f413c004d90 0x7f413c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f4152ffd700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f413c003a00 0x7f413c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41527fc700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 0x7f413c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41537fe700 1 -- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 msgr2=0x7f413c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41537fe700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41540ceda0 0x7f413c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41537fe700 1 -- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f413c003a00 msgr2=0x7f413c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41537fe700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f413c003a00 0x7f413c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41537fe700 1 -- 172.21.15.89:0/1772315828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c0035b0 con 0x7f413c004d90 2023-07-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f41537fe700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f413c004d90 0x7f413c005200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f414c009fc0 tx=0x7f413c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b955e55c87356d89 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c00ebd0 con 0x7f413c004d90 2023-07-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c00f040 con 0x7f413c004d90 2023-07-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.964+0000 7f4151ffb700 1 -- 172.21.15.89:0/1772315828 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c014480 con 0x7f413c004d90 2023-07-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.965+0000 7f4159363700 1 -- 172.21.15.89:0/1772315828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f413c004d90 msgr2=0x7f413c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.965+0000 7f4159363700 1 --2- 172.21.15.89:0/1772315828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f413c004d90 0x7f413c005200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f414c009fc0 tx=0x7f413c005880 comp rx=0 tx=0).stop 2023-07-06T02:23:00.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.965+0000 7f41537fe700 1 -- 172.21.15.89:0/1772315828 reap_dead start 2023-07-06T02:23:00.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.965+0000 7f4159363700 1 -- 172.21.15.89:0/1772315828 shutdown_connections 2023-07-06T02:23:00.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.965+0000 7f4159363700 1 -- 172.21.15.89:0/1772315828 >> 172.21.15.89:0/1772315828 conn(0x7f41540b5390 msgr2=0x7f41540d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:00.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.965+0000 7f4159363700 1 -- 172.21.15.89:0/1772315828 shutdown_connections 2023-07-06T02:23:00.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.965+0000 7f4159363700 1 -- 172.21.15.89:0/1772315828 wait complete. 2023-07-06T02:23:00.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.966+0000 7f4159363700 1 Processor -- start 2023-07-06T02:23:00.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.966+0000 7f4159363700 1 -- start start 2023-07-06T02:23:00.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4159363700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4154148400 0x7f4154158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4159363700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41540b7080 0x7f41540c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4159363700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41540b5a20 0x7f41540c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4159363700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41540b5fe0 con 0x7f41540b7080 2023-07-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4159363700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41540b62c0 con 0x7f41540b5a20 2023-07-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4159363700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540b65a0 con 0x7f4154148400 2023-07-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41540b7080 0x7f41540c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41540b7080 0x7f41540c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60370/0 (socket says 172.21.15.89:60370) 2023-07-06T02:23:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 -- 172.21.15.89:0/3532571216 learned_addr learned my addr 172.21.15.89:0/3532571216 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4152ffd700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4154148400 0x7f4154158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41537fe700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41540b5a20 0x7f41540c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 -- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4154148400 msgr2=0x7f4154158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4154148400 0x7f4154158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 -- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41540b5a20 msgr2=0x7f41540c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41540b5a20 0x7f41540c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 -- 172.21.15.89:0/3532571216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41540b6880 con 0x7f41540b7080 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41537fe700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41540b5a20 0x7f41540c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f41527fc700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41540b7080 0x7f41540c66f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f41480017a0 tx=0x7f41480076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f413b7fe700 1 -- 172.21.15.89:0/3532571216 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41480077c0 con 0x7f41540b7080 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f413b7fe700 1 -- 172.21.15.89:0/3532571216 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414800f8d0 con 0x7f41540b7080 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41540b6b60 con 0x7f41540b7080 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.967+0000 7f413b7fe700 1 -- 172.21.15.89:0/3532571216 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414800fc80 con 0x7f41540b7080 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.968+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41541590f0 con 0x7f41540b7080 2023-07-06T02:23:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.969+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41540b6d50 con 0x7f41540b7080 2023-07-06T02:23:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.969+0000 7f413b7fe700 1 -- 172.21.15.89:0/3532571216 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f414801d460 con 0x7f41540b7080 2023-07-06T02:23:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.969+0000 7f413b7fe700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f414006ace0 0x7f414006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.969+0000 7f4152ffd700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f414006ace0 0x7f414006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.969+0000 7f413b7fe700 1 -- 172.21.15.89:0/3532571216 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4148089850 con 0x7f41540b7080 2023-07-06T02:23:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.970+0000 7f4152ffd700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f414006ace0 0x7f414006d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41540b18e0 tx=0x7f4144009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:00.974+0000 7f413b7fe700 1 -- 172.21.15.89:0/3532571216 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41480580a0 con 0x7f41540b7080 2023-07-06T02:23:01.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.207+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4154074270 con 0x7f41540b7080 2023-07-06T02:23:01.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.207+0000 7f413b7fe700 1 -- 172.21.15.89:0/3532571216 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4148057c30 con 0x7f41540b7080 2023-07-06T02:23:01.209 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-06T02:23:01.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f414006ace0 msgr2=0x7f414006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:01.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f414006ace0 0x7f414006d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41540b18e0 tx=0x7f4144009210 comp rx=0 tx=0).stop 2023-07-06T02:23:01.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41540b7080 msgr2=0x7f41540c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:01.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41540b7080 0x7f41540c66f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f41480017a0 tx=0x7f41480076a0 comp rx=0 tx=0).stop 2023-07-06T02:23:01.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 shutdown_connections 2023-07-06T02:23:01.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41540b5a20 0x7f41540c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:01.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4154148400 0x7f4154158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:01.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f414006ace0 0x7f414006d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:01.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 --2- 172.21.15.89:0/3532571216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41540b7080 0x7f41540c66f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:01.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 >> 172.21.15.89:0/3532571216 conn(0x7f41540b5390 msgr2=0x7f41540d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:01.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 shutdown_connections 2023-07-06T02:23:01.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:01.209+0000 7f4159363700 1 -- 172.21.15.89:0/3532571216 wait complete. 2023-07-06T02:23:01.223 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-06T02:23:01.223 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T02:23:01.224 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-07-06T02:23:01.224 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-06T02:23:01.224 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:01.224 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-06T02:23:01.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T02:23:01.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T02:23:01.225 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-06T02:23:01.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.226 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-07-06T02:23:01.242 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:01.244 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:01.244 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:01.244 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:01.245 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T02:23:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T02:23:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42058sec preferred_lft 42058sec 2023-07-06T02:23:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5513:ae8f:6526:fe98/64 scope link noprefixroute 2023-07-06T02:23:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:01.246 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T02:23:01.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.246 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:01.247 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:01.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2023-07-06T02:23:01.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2023-07-06T02:23:01.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2023-07-06T02:23:01.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T02:23:01.248 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:01.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.403 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T02:23:01.431 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-07-06T02:23:01.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.433 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-07-06T02:23:01.509 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-07-06T02:23:01.509 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T02:23:01.509 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T02:23:01.510 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T02:23:01.510 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T02:23:01.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.511 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:01.511 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:01.511 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T02:23:01.511 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T02:23:01.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T02:23:01.512 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:01.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.757 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-07-06T02:23:01.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.773 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-07-06T02:23:01.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.830 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:01.830 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:01.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-06T02:23:01.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-06T02:23:01.831 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:01.923 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-06T02:23:01.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:01.924 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:01.925 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:01.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-06T02:23:01.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T02:23:01.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-06T02:23:01.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-06T02:23:01.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-06T02:23:01.927 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:02.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:02.136 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:02.137 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:02.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-07-06T02:23:02.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T02:23:02.138 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:02.238 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-06T02:23:02.238 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:02.238 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:02.258 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-06T02:23:02.258 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:02.259 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:02.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2023-07-06T02:23:02.381 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T02:23:02.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T02:23:02.447 INFO:teuthology.orchestra.run.smithi089.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T02:23:02.448 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T02:23:02.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:02.449 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2023-07-06T02:23:02.505 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T02:23:02.506 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-06T02:23:02.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2023-07-06T02:23:02.573 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T02:23:02.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T02:23:02.639 INFO:teuthology.orchestra.run.smithi089.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T02:23:02.640 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T02:23:02.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:02.642 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2023-07-06T02:23:02.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:2023-07-06T02:23:02.640+0000 7f6007fe55c0 -1 init, newargv = 0x56515568e1e0 newargc=15 2023-07-06T02:23:02.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[139278]: starting ceph client 2023-07-06T02:23:02.648 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[139278]: starting fuse 2023-07-06T02:23:02.659 INFO:teuthology.orchestra.run.smithi089.stdout:46 2023-07-06T02:23:02.659 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-07-06T02:23:02.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T02:23:02.660 DEBUG:teuthology.orchestra.run.smithi089:> import glob 2023-07-06T02:23:02.660 DEBUG:teuthology.orchestra.run.smithi089:> import re 2023-07-06T02:23:02.661 DEBUG:teuthology.orchestra.run.smithi089:> import os 2023-07-06T02:23:02.661 DEBUG:teuthology.orchestra.run.smithi089:> import subprocess 2023-07-06T02:23:02.662 DEBUG:teuthology.orchestra.run.smithi089:> 2023-07-06T02:23:02.662 DEBUG:teuthology.orchestra.run.smithi089:> def _find_admin_socket(client_name): 2023-07-06T02:23:02.662 DEBUG:teuthology.orchestra.run.smithi089:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-06T02:23:02.662 DEBUG:teuthology.orchestra.run.smithi089:> files = glob.glob(asok_path) 2023-07-06T02:23:02.663 DEBUG:teuthology.orchestra.run.smithi089:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-06T02:23:02.663 DEBUG:teuthology.orchestra.run.smithi089:> 2023-07-06T02:23:02.663 DEBUG:teuthology.orchestra.run.smithi089:> # Given a non-glob path, it better be there 2023-07-06T02:23:02.664 DEBUG:teuthology.orchestra.run.smithi089:> if "*" not in asok_path: 2023-07-06T02:23:02.664 DEBUG:teuthology.orchestra.run.smithi089:> assert(len(files) == 1) 2023-07-06T02:23:02.664 DEBUG:teuthology.orchestra.run.smithi089:> return files[0] 2023-07-06T02:23:02.665 DEBUG:teuthology.orchestra.run.smithi089:> 2023-07-06T02:23:02.665 DEBUG:teuthology.orchestra.run.smithi089:> for f in files: 2023-07-06T02:23:02.665 DEBUG:teuthology.orchestra.run.smithi089:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T02:23:02.665 DEBUG:teuthology.orchestra.run.smithi089:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T02:23:02.666 DEBUG:teuthology.orchestra.run.smithi089:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T02:23:02.666 DEBUG:teuthology.orchestra.run.smithi089:> contents = proc_f.read() 2023-07-06T02:23:02.666 DEBUG:teuthology.orchestra.run.smithi089:> if mountpoint in contents: 2023-07-06T02:23:02.667 DEBUG:teuthology.orchestra.run.smithi089:> return f 2023-07-06T02:23:02.667 DEBUG:teuthology.orchestra.run.smithi089:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T02:23:02.667 DEBUG:teuthology.orchestra.run.smithi089:> 2023-07-06T02:23:02.667 DEBUG:teuthology.orchestra.run.smithi089:> print(_find_admin_socket("client.admin")) 2023-07-06T02:23:02.668 DEBUG:teuthology.orchestra.run.smithi089:> ' 2023-07-06T02:23:02.782 INFO:teuthology.orchestra.run.smithi089.stdout:/var/run/ceph/ceph-client.admin.139278.asok 2023-07-06T02:23:02.965 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.139278.asok 2023-07-06T02:23:02.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:02.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.139278.asok status 2023-07-06T02:23:03.122 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-07-06T02:23:03.123 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": { 2023-07-06T02:23:03.123 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T02:23:03.123 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T02:23:03.123 INFO:teuthology.orchestra.run.smithi089.stdout: "entity_id": "admin", 2023-07-06T02:23:03.124 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": "smithi089", 2023-07-06T02:23:03.124 INFO:teuthology.orchestra.run.smithi089.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-06T02:23:03.124 INFO:teuthology.orchestra.run.smithi089.stdout: "pid": "139278", 2023-07-06T02:23:03.124 INFO:teuthology.orchestra.run.smithi089.stdout: "root": "/" 2023-07-06T02:23:03.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:23:03.124 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_count": 0, 2023-07-06T02:23:03.124 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_pinned_count": 0, 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4473, 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: "inst": { 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: "name": { 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "client", 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4473 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2023-07-06T02:23:03.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-07-06T02:23:03.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2023-07-06T02:23:03.126 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1196725985 2023-07-06T02:23:03.126 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:23:03.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:23:03.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2023-07-06T02:23:03.126 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-07-06T02:23:03.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1196725985 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: "inst_str": "client.4473 192.168.0.1:0/1196725985", 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr_str": "192.168.0.1:0/1196725985", 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: "inode_count": 1, 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_epoch": 6, 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch": 24, 2023-07-06T02:23:03.127 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch_barrier": 0, 2023-07-06T02:23:03.128 INFO:teuthology.orchestra.run.smithi089.stdout: "blocklisted": false, 2023-07-06T02:23:03.128 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs" 2023-07-06T02:23:03.128 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-07-06T02:23:03.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:03.133 DEBUG:teuthology.orchestra.run.smithi089:> stat --file-system '--printf=%T 2023-07-06T02:23:03.133 DEBUG:teuthology.orchestra.run.smithi089:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:03.149 INFO:teuthology.orchestra.run.smithi089.stdout:fuseblk 2023-07-06T02:23:03.150 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:03.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:03.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:03.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:03.217 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-06T02:23:03.325 INFO:teuthology.orchestra.run.smithi089.stdout:# file: . 2023-07-06T02:23:03.325 INFO:teuthology.orchestra.run.smithi089.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-06T02:23:03.326 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:03.326 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:03.326 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:03.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:03.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.503+0000 7f5cf06fd700 1 Processor -- start 2023-07-06T02:23:03.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.503+0000 7f5cf06fd700 1 -- start start 2023-07-06T02:23:03.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cf06fd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ce80cecf0 0x7f5ce80ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cf06fd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cf06fd700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce806ebe0 con 0x7f5ce806e250 2023-07-06T02:23:03.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cf06fd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce80caa30 con 0x7f5ce80cecf0 2023-07-06T02:23:03.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cf06fd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce806abe0 con 0x7f5ce80ca5f0 2023-07-06T02:23:03.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cee499700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5ce80cecf0 0x7f5ce80ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cedc98700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5ce806e250 0x7f5ce80c9ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57066/0 (socket says 172.21.15.89:57066) 2023-07-06T02:23:03.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cedc98700 1 -- 172.21.15.89:0/1761014271 learned_addr learned my addr 172.21.15.89:0/1761014271 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5ceec9a700 1 --2- 172.21.15.89:0/1761014271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cee499700 1 -- 172.21.15.89:0/1761014271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 msgr2=0x7f5ce806e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cee499700 1 --2- 172.21.15.89:0/1761014271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce806e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5ced497700 1 -- 172.21.15.89:0/1761014271 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807916752 0 0) 0x7f5cd8001110 con 0x7f5ce806e250 2023-07-06T02:23:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cee499700 1 -- 172.21.15.89:0/1761014271 >> v1:172.21.15.89:6789/0 conn(0x7f5ce806e250 legacy=0x7f5ce80c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cee499700 1 -- 172.21.15.89:0/1761014271 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce806b160 con 0x7f5ce80cecf0 2023-07-06T02:23:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5cee499700 1 --2- 172.21.15.89:0/1761014271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce80ccd90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009840 tx=0x7f5ce4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469331bab9001e66 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.504+0000 7f5ced497700 1 -- 172.21.15.89:0/1761014271 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4011830 con 0x7f5ce80cecf0 2023-07-06T02:23:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5ced497700 1 -- 172.21.15.89:0/1761014271 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4011990 con 0x7f5ce80cecf0 2023-07-06T02:23:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5ced497700 1 -- 172.21.15.89:0/1761014271 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4011d20 con 0x7f5ce80cecf0 2023-07-06T02:23:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 -- 172.21.15.89:0/1761014271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 msgr2=0x7f5ce80ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/1761014271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce80ccd90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009840 tx=0x7f5ce4009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:03.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 -- 172.21.15.89:0/1761014271 shutdown_connections 2023-07-06T02:23:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/1761014271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce806e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/1761014271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce80ccd90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 -- 172.21.15.89:0/1761014271 >> 172.21.15.89:0/1761014271 conn(0x7f5ce80cf1c0 msgr2=0x7f5ce80cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 -- 172.21.15.89:0/1761014271 shutdown_connections 2023-07-06T02:23:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.505+0000 7f5cf06fd700 1 -- 172.21.15.89:0/1761014271 wait complete. 2023-07-06T02:23:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.507+0000 7f5cf06fd700 1 Processor -- start 2023-07-06T02:23:03.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.507+0000 7f5cf06fd700 1 -- start start 2023-07-06T02:23:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce806e250 0x7f5ce814fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce8155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce8155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81ef000 con 0x7f5ce80cecf0 2023-07-06T02:23:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80c8a80 con 0x7f5ce806e250 2023-07-06T02:23:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80b3f10 con 0x7f5ce80ca5f0 2023-07-06T02:23:03.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce8155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce8155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60404/0 (socket says 172.21.15.89:60404) 2023-07-06T02:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 -- 172.21.15.89:0/3448372116 learned_addr learned my addr 172.21.15.89:0/3448372116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cee499700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce806e250 0x7f5ce814fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cedc98700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce8155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 -- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 msgr2=0x7f5ce8155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce8155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 -- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce806e250 msgr2=0x7f5ce814fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce806e250 0x7f5ce814fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 -- 172.21.15.89:0/3448372116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40094a0 con 0x7f5ce80cecf0 2023-07-06T02:23:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5ceec9a700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce8155b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000ceb0 tx=0x7f5ce000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cdeffd700 1 -- 172.21.15.89:0/3448372116 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0011070 con 0x7f5ce80cecf0 2023-07-06T02:23:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cdeffd700 1 -- 172.21.15.89:0/3448372116 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce000c2e0 con 0x7f5ce80cecf0 2023-07-06T02:23:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce80b2010 con 0x7f5ce80cecf0 2023-07-06T02:23:03.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cdeffd700 1 -- 172.21.15.89:0/3448372116 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0003c40 con 0x7f5ce80cecf0 2023-07-06T02:23:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.508+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce8150530 con 0x7f5ce80cecf0 2023-07-06T02:23:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.510+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce81efff0 con 0x7f5ce80cecf0 2023-07-06T02:23:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.510+0000 7f5cdeffd700 1 -- 172.21.15.89:0/3448372116 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ce000e9f0 con 0x7f5ce80cecf0 2023-07-06T02:23:03.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.510+0000 7f5cdeffd700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ccc06ace0 0x7f5ccc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.510+0000 7f5cee499700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ccc06ace0 0x7f5ccc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.511+0000 7f5cdeffd700 1 -- 172.21.15.89:0/3448372116 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ce008b550 con 0x7f5ce80cecf0 2023-07-06T02:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.511+0000 7f5cee499700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ccc06ace0 0x7f5ccc06d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40090b0 tx=0x7f5ce401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:03.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.515+0000 7f5cdeffd700 1 -- 172.21.15.89:0/3448372116 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5ce00563a0 con 0x7f5ce80cecf0 2023-07-06T02:23:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.703+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ce8074270 con 0x7f5ce80cecf0 2023-07-06T02:23:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.704+0000 7f5cdeffd700 1 -- 172.21.15.89:0/3448372116 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f5ce00599c0 con 0x7f5ce80cecf0 2023-07-06T02:23:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:03.707 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:03.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ccc06ace0 msgr2=0x7f5ccc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ccc06ace0 0x7f5ccc06d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40090b0 tx=0x7f5ce401c040 comp rx=0 tx=0).stop 2023-07-06T02:23:03.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 msgr2=0x7f5ce8155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce8155b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000ceb0 tx=0x7f5ce000cf90 comp rx=0 tx=0).stop 2023-07-06T02:23:03.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 shutdown_connections 2023-07-06T02:23:03.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ccc06ace0 0x7f5ccc06d190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ce80cecf0 0x7f5ce8155b60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce806e250 0x7f5ce814fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 --2- 172.21.15.89:0/3448372116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce80ca5f0 0x7f5ce8155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 >> 172.21.15.89:0/3448372116 conn(0x7f5ce80cf1c0 msgr2=0x7f5ce815ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:03.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 shutdown_connections 2023-07-06T02:23:03.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.706+0000 7f5cf06fd700 1 -- 172.21.15.89:0/3448372116 wait complete. 2023-07-06T02:23:03.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:03.721 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.842+0000 7f002818c700 1 Processor -- start 2023-07-06T02:23:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f002818c700 1 -- start start 2023-07-06T02:23:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f002818c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f0020160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f002818c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0020160bb0 0x7f00200b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f002818c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00201600b0 0x7f00200b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f002818c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00200cb3d0 con 0x7f0020160bb0 2023-07-06T02:23:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f002818c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0020156e80 con 0x7f00201600b0 2023-07-06T02:23:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f002818c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200ca370 con 0x7f0020156ab0 2023-07-06T02:23:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.843+0000 7f0025727700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0020160bb0 0x7f00200b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0020160bb0 0x7f00200b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60410/0 (socket says 172.21.15.89:60410) 2023-07-06T02:23:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 -- 172.21.15.89:0/2946348530 learned_addr learned my addr 172.21.15.89:0/2946348530 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0026729700 1 --2- 172.21.15.89:0/2946348530 >> v2:172.21.15.172:3300/0 conn(0x7f00201600b0 0x7f00200b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025f28700 1 --2- 172.21.15.89:0/2946348530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f0020160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 -- 172.21.15.89:0/2946348530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 msgr2=0x7f0020160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 --2- 172.21.15.89:0/2946348530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f0020160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 -- 172.21.15.89:0/2946348530 >> v2:172.21.15.172:3300/0 conn(0x7f00201600b0 msgr2=0x7f00200b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 --2- 172.21.15.89:0/2946348530 >> v2:172.21.15.172:3300/0 conn(0x7f00201600b0 0x7f00200b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 -- 172.21.15.89:0/2946348530 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200ca610 con 0x7f0020160bb0 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025f28700 1 --2- 172.21.15.89:0/2946348530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f0020160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0026729700 1 --2- 172.21.15.89:0/2946348530 >> v2:172.21.15.172:3300/0 conn(0x7f00201600b0 0x7f00200b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0025727700 1 --2- 172.21.15.89:0/2946348530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0020160bb0 0x7f00200b1490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f002015fef0 tx=0x7f001c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ae088a9aae0596 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0024f26700 1 -- 172.21.15.89:0/2946348530 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00b870 con 0x7f0020160bb0 2023-07-06T02:23:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0024f26700 1 -- 172.21.15.89:0/2946348530 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c00b9d0 con 0x7f0020160bb0 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.844+0000 7f0024f26700 1 -- 172.21.15.89:0/2946348530 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00bd60 con 0x7f0020160bb0 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 -- 172.21.15.89:0/2946348530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0020160bb0 msgr2=0x7f00200b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 --2- 172.21.15.89:0/2946348530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0020160bb0 0x7f00200b1490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f002015fef0 tx=0x7f001c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 -- 172.21.15.89:0/2946348530 shutdown_connections 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 --2- 172.21.15.89:0/2946348530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f0020160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 --2- 172.21.15.89:0/2946348530 >> v2:172.21.15.172:3300/0 conn(0x7f00201600b0 0x7f00200b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 --2- 172.21.15.89:0/2946348530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0020160bb0 0x7f00200b1490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 -- 172.21.15.89:0/2946348530 >> 172.21.15.89:0/2946348530 conn(0x7f00200cc490 msgr2=0x7f00200cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 -- 172.21.15.89:0/2946348530 shutdown_connections 2023-07-06T02:23:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.845+0000 7f002818c700 1 -- 172.21.15.89:0/2946348530 wait complete. 2023-07-06T02:23:03.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.846+0000 7f002818c700 1 Processor -- start 2023-07-06T02:23:03.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.846+0000 7f002818c700 1 -- start start 2023-07-06T02:23:03.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f002818c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f00200c3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f002818c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00201600b0 0x7f0020155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f002818c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00200adc30 0x7f00200b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f002818c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200ae6d0 con 0x7f00200adc30 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f002818c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200ae110 con 0x7f00201600b0 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f002818c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200ae3f0 con 0x7f0020156ab0 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00200adc30 0x7f00200b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00200adc30 0x7f00200b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60418/0 (socket says 172.21.15.89:60418) 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 -- 172.21.15.89:0/2239173808 learned_addr learned my addr 172.21.15.89:0/2239173808 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0025f28700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f00200c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0025727700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00201600b0 0x7f0020155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 -- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 msgr2=0x7f00200c3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f00200c3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 -- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00201600b0 msgr2=0x7f0020155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00201600b0 0x7f0020155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 -- 172.21.15.89:0/2239173808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c009550 con 0x7f00200adc30 2023-07-06T02:23:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0025f28700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f00200c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f0026729700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00200adc30 0x7f00200b38b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f001800cea0 tx=0x7f001800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f00167fc700 1 -- 172.21.15.89:0/2239173808 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001800ae50 con 0x7f00200adc30 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f00167fc700 1 -- 172.21.15.89:0/2239173808 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018003a20 con 0x7f00200adc30 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f00167fc700 1 -- 172.21.15.89:0/2239173808 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018003dd0 con 0x7f00200adc30 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.847+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00200c3f00 con 0x7f00200adc30 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.848+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0020156430 con 0x7f00200adc30 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.848+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00200c3d60 con 0x7f00200adc30 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.849+0000 7f00167fc700 1 -- 172.21.15.89:0/2239173808 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0018003f30 con 0x7f00200adc30 2023-07-06T02:23:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.849+0000 7f00167fc700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f000406ace0 0x7f000406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.849+0000 7f00167fc700 1 -- 172.21.15.89:0/2239173808 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0018089bd0 con 0x7f00200adc30 2023-07-06T02:23:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.849+0000 7f0025f28700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f000406ace0 0x7f000406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.850+0000 7f0025f28700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f000406ace0 0x7f000406d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0010006fd0 tx=0x7f0010008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:03.854+0000 7f00167fc700 1 -- 172.21.15.89:0/2239173808 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f00200c3d60 con 0x7f00200adc30 2023-07-06T02:23:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.040+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00200c4420 con 0x7f00200adc30 2023-07-06T02:23:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.040+0000 7f00167fc700 1 -- 172.21.15.89:0/2239173808 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f00200c4420 con 0x7f00200adc30 2023-07-06T02:23:04.042 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f000406ace0 msgr2=0x7f000406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f000406ace0 0x7f000406d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0010006fd0 tx=0x7f0010008040 comp rx=0 tx=0).stop 2023-07-06T02:23:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00200adc30 msgr2=0x7f00200b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00200adc30 0x7f00200b38b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f001800cea0 tx=0x7f001800cf80 comp rx=0 tx=0).stop 2023-07-06T02:23:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 shutdown_connections 2023-07-06T02:23:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f000406ace0 0x7f000406d190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00200adc30 0x7f00200b38b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0020156ab0 0x7f00200c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 --2- 172.21.15.89:0/2239173808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00201600b0 0x7f0020155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 >> 172.21.15.89:0/2239173808 conn(0x7f00200cc490 msgr2=0x7f00200cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 shutdown_connections 2023-07-06T02:23:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.042+0000 7f002818c700 1 -- 172.21.15.89:0/2239173808 wait complete. 2023-07-06T02:23:04.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.178+0000 7f11e0165700 1 Processor -- start 2023-07-06T02:23:04.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.178+0000 7f11e0165700 1 -- start start 2023-07-06T02:23:04.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11e0165700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f11d815a3e0 0x7f11d80caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11e0165700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11d8071630 0x7f11d815dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11e0165700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d806a500 con 0x7f11d806bcd0 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11e0165700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d806ec60 con 0x7f11d815a3e0 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11e0165700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d806f190 con 0x7f11d8071630 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f11d815a3e0 0x7f11d80caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f11d815a3e0 0x7f11d80caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60422/0 (socket says 172.21.15.89:60422) 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 -- 172.21.15.89:0/820832501 learned_addr learned my addr 172.21.15.89:0/820832501 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11de702700 1 --2- 172.21.15.89:0/820832501 >> v2:172.21.15.172:3300/0 conn(0x7f11d8071630 0x7f11d815dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 -- 172.21.15.89:0/820832501 >> v2:172.21.15.172:3300/0 conn(0x7f11d8071630 msgr2=0x7f11d815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 --2- 172.21.15.89:0/820832501 >> v2:172.21.15.172:3300/0 conn(0x7f11d8071630 0x7f11d815dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 -- 172.21.15.89:0/820832501 >> v1:172.21.15.172:6789/0 conn(0x7f11d806bcd0 legacy=0x7f11d8070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 -- 172.21.15.89:0/820832501 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d80cb980 con 0x7f11d815a3e0 2023-07-06T02:23:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11ddf01700 1 --2- 172.21.15.89:0/820832501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d815a3e0 0x7f11d80caf80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11d80d0270 tx=0x7f11d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f000f819df40cdd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11dceff700 1 -- 172.21.15.89:0/820832501 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d400b870 con 0x7f11d815a3e0 2023-07-06T02:23:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11dceff700 1 -- 172.21.15.89:0/820832501 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d400b9d0 con 0x7f11d815a3e0 2023-07-06T02:23:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.179+0000 7f11dceff700 1 -- 172.21.15.89:0/820832501 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d400bd60 con 0x7f11d815a3e0 2023-07-06T02:23:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 -- 172.21.15.89:0/820832501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d815a3e0 msgr2=0x7f11d80caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 --2- 172.21.15.89:0/820832501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d815a3e0 0x7f11d80caf80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11d80d0270 tx=0x7f11d4009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 -- 172.21.15.89:0/820832501 shutdown_connections 2023-07-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 --2- 172.21.15.89:0/820832501 >> v2:172.21.15.172:3300/0 conn(0x7f11d8071630 0x7f11d815dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 --2- 172.21.15.89:0/820832501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d815a3e0 0x7f11d80caf80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 -- 172.21.15.89:0/820832501 >> 172.21.15.89:0/820832501 conn(0x7f11d80cf1c0 msgr2=0x7f11d80cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 -- 172.21.15.89:0/820832501 shutdown_connections 2023-07-06T02:23:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.180+0000 7f11e0165700 1 -- 172.21.15.89:0/820832501 wait complete. 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.181+0000 7f11e0165700 1 Processor -- start 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.181+0000 7f11e0165700 1 -- start start 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11e0165700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d806bcd0 0x7f11d80c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11e0165700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11d8071630 0x7f11d8151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11e0165700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11d815a3e0 0x7f11d81e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11e0165700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d8151770 con 0x7f11d806bcd0 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11e0165700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81e62b0 con 0x7f11d8071630 2023-07-06T02:23:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11e0165700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80c3d70 con 0x7f11d815a3e0 2023-07-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11ddf01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d806bcd0 0x7f11d80c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11ddf01700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d806bcd0 0x7f11d80c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60428/0 (socket says 172.21.15.89:60428) 2023-07-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11ddf01700 1 -- 172.21.15.89:0/1587609332 learned_addr learned my addr 172.21.15.89:0/1587609332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11ddf01700 1 -- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11d815a3e0 msgr2=0x7f11d81e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11de702700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11d815a3e0 0x7f11d81e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11dd700700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11d8071630 0x7f11d8151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11ddf01700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11d815a3e0 0x7f11d81e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11ddf01700 1 -- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11d8071630 msgr2=0x7f11d8151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.182+0000 7f11ddf01700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11d8071630 0x7f11d8151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11ddf01700 1 -- 172.21.15.89:0/1587609332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4009550 con 0x7f11d806bcd0 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11de702700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11d815a3e0 0x7f11d81e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11dd700700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11d8071630 0x7f11d8151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11ddf01700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d806bcd0 0x7f11d80c36d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11ce7fc700 1 -- 172.21.15.89:0/1587609332 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4004540 con 0x7f11d806bcd0 2023-07-06T02:23:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11ce7fc700 1 -- 172.21.15.89:0/1587609332 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d40036a0 con 0x7f11d806bcd0 2023-07-06T02:23:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d80c4050 con 0x7f11d806bcd0 2023-07-06T02:23:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11ce7fc700 1 -- 172.21.15.89:0/1587609332 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4003a30 con 0x7f11d806bcd0 2023-07-06T02:23:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.183+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d80c4610 con 0x7f11d806bcd0 2023-07-06T02:23:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.184+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d8074270 con 0x7f11d806bcd0 2023-07-06T02:23:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.184+0000 7f11ce7fc700 1 -- 172.21.15.89:0/1587609332 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11d4021070 con 0x7f11d806bcd0 2023-07-06T02:23:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.185+0000 7f11ce7fc700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f11bc06ace0 0x7f11bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.185+0000 7f11ce7fc700 1 -- 172.21.15.89:0/1587609332 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f11d408aee0 con 0x7f11d806bcd0 2023-07-06T02:23:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.185+0000 7f11dd700700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f11bc06ace0 0x7f11bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.185+0000 7f11dd700700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f11bc06ace0 0x7f11bc06d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11d81e5aa0 tx=0x7f11c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.189+0000 7f11ce7fc700 1 -- 172.21.15.89:0/1587609332 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11d4055d30 con 0x7f11d806bcd0 2023-07-06T02:23:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.374+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11d80cb2a0 con 0x7f11d806bcd0 2023-07-06T02:23:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.374+0000 7f11ce7fc700 1 -- 172.21.15.89:0/1587609332 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11d40150a0 con 0x7f11d806bcd0 2023-07-06T02:23:04.376 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.376+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f11bc06ace0 msgr2=0x7f11bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.376+0000 7f11e0165700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f11bc06ace0 0x7f11bc06d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11d81e5aa0 tx=0x7f11c8009040 comp rx=0 tx=0).stop 2023-07-06T02:23:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.376+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d806bcd0 msgr2=0x7f11d80c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.376+0000 7f11e0165700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d806bcd0 0x7f11d80c36d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d4004750 comp rx=0 tx=0).stop 2023-07-06T02:23:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 shutdown_connections 2023-07-06T02:23:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11d815a3e0 0x7f11d81e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f11bc06ace0 0x7f11bc06d190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11d806bcd0 0x7f11d80c36d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 --2- 172.21.15.89:0/1587609332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11d8071630 0x7f11d8151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 >> 172.21.15.89:0/1587609332 conn(0x7f11d80cf1c0 msgr2=0x7f11d80b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 shutdown_connections 2023-07-06T02:23:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.377+0000 7f11e0165700 1 -- 172.21.15.89:0/1587609332 wait complete. 2023-07-06T02:23:04.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:04.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.511+0000 7fa3f56f6700 1 Processor -- start 2023-07-06T02:23:04.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.512+0000 7fa3f56f6700 1 -- start start 2023-07-06T02:23:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.512+0000 7fa3f56f6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 0x7fa3f00ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.512+0000 7fa3f56f6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f0160ee0 con 0x7fa3f00cadf0 2023-07-06T02:23:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.512+0000 7fa3f56f6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f015d930 con 0x7fa3f00aedc0 2023-07-06T02:23:04.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.512+0000 7fa3f56f6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01539a0 con 0x7fa3f0071da0 2023-07-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.512+0000 7fa3ef7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa3f00cadf0 0x7fa3f015cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57110/0 (socket says 172.21.15.89:57110) 2023-07-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.512+0000 7fa3ef7fe700 1 -- 172.21.15.89:0/3018489730 learned_addr learned my addr 172.21.15.89:0/3018489730 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ee7fc700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 0x7fa3f00ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022753077 0 0) 0x7fa3f0160ee0 con 0x7fa3f00cadf0 2023-07-06T02:23:04.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d80035b0 con 0x7fa3f00cadf0 2023-07-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1088176482 0 0) 0x7fa3d80035b0 con 0x7fa3f00cadf0 2023-07-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 msgr2=0x7fa3f00ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 0x7fa3f00ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 >> v1:172.21.15.172:6789/0 conn(0x7fa3f00aedc0 legacy=0x7fa3f0071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0160120 con 0x7fa3f00cadf0 2023-07-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fa3e80019f0 con 0x7fa3f00cadf0 2023-07-06T02:23:04.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 >> v1:172.21.15.89:6789/0 conn(0x7fa3f00cadf0 legacy=0x7fa3f015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 0x7fa3d8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d8003fb0 0x7fa3d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d8004d90 0x7fa3d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01539a0 con 0x7fa3d8004d90 2023-07-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d800baf0 con 0x7fa3d8003fb0 2023-07-06T02:23:04.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d800bcb0 con 0x7fa3f0071da0 2023-07-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ef7fe700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d8004d90 0x7fa3d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3eeffd700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 0x7fa3d8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ee7fc700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d8003fb0 0x7fa3d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ef7fe700 1 -- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 msgr2=0x7fa3d8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ef7fe700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f0071da0 0x7fa3d8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ef7fe700 1 -- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d8003fb0 msgr2=0x7fa3d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:04.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ef7fe700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d8003fb0 0x7fa3d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.513+0000 7fa3ef7fe700 1 -- 172.21.15.89:0/3018489730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d80035b0 con 0x7fa3d8004d90 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3ef7fe700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d8004d90 0x7fa3d8005200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800a800 tx=0x7fa3d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d27936f0df3657 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800a2f0 con 0x7fa3d8004d90 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8012600 con 0x7fa3d8004d90 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3edffb700 1 -- 172.21.15.89:0/3018489730 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80129b0 con 0x7fa3d8004d90 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3f56f6700 1 -- 172.21.15.89:0/3018489730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d8004d90 msgr2=0x7fa3d8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3f56f6700 1 --2- 172.21.15.89:0/3018489730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3d8004d90 0x7fa3d8005200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800a800 tx=0x7fa3d8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:04.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3ef7fe700 1 -- 172.21.15.89:0/3018489730 reap_dead start 2023-07-06T02:23:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3f56f6700 1 -- 172.21.15.89:0/3018489730 shutdown_connections 2023-07-06T02:23:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3f56f6700 1 -- 172.21.15.89:0/3018489730 >> 172.21.15.89:0/3018489730 conn(0x7fa3f00cd0f0 msgr2=0x7fa3f00cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3f56f6700 1 -- 172.21.15.89:0/3018489730 shutdown_connections 2023-07-06T02:23:04.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.514+0000 7fa3f56f6700 1 -- 172.21.15.89:0/3018489730 wait complete. 2023-07-06T02:23:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 Processor -- start 2023-07-06T02:23:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 -- start start 2023-07-06T02:23:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d8004d90 0x7fa3f0150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f0151290 0x7fa3f00c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f00c8690 0x7fa3f00b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00c8ba0 con 0x7fa3f0151290 2023-07-06T02:23:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00c8e80 con 0x7fa3f00c8690 2023-07-06T02:23:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3f56f6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00c9160 con 0x7fa3d8004d90 2023-07-06T02:23:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.516+0000 7fa3ee7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f0151290 0x7fa3f00c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f0151290 0x7fa3f00c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60450/0 (socket says 172.21.15.89:60450) 2023-07-06T02:23:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 -- 172.21.15.89:0/1004674908 learned_addr learned my addr 172.21.15.89:0/1004674908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ef7fe700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f00c8690 0x7fa3f00b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3eeffd700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d8004d90 0x7fa3f0150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 -- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d8004d90 msgr2=0x7fa3f0150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d8004d90 0x7fa3f0150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 -- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f00c8690 msgr2=0x7fa3f00b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f00c8690 0x7fa3f00b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 -- 172.21.15.89:0/1004674908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00c9440 con 0x7fa3f0151290 2023-07-06T02:23:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3eeffd700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d8004d90 0x7fa3f0150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ef7fe700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f00c8690 0x7fa3f00b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3ee7fc700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f0151290 0x7fa3f00c8150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00119c0 tx=0x7fa3e0011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3d77fe700 1 -- 172.21.15.89:0/1004674908 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004240 con 0x7fa3f0151290 2023-07-06T02:23:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3d77fe700 1 -- 172.21.15.89:0/1004674908 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e00043a0 con 0x7fa3f0151290 2023-07-06T02:23:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f00c9720 con 0x7fa3f0151290 2023-07-06T02:23:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3d77fe700 1 -- 172.21.15.89:0/1004674908 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000f730 con 0x7fa3f0151290 2023-07-06T02:23:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.517+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f00c9c40 con 0x7fa3f0151290 2023-07-06T02:23:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.518+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f00c9a90 con 0x7fa3f0151290 2023-07-06T02:23:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.518+0000 7fa3d77fe700 1 -- 172.21.15.89:0/1004674908 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3e000f890 con 0x7fa3f0151290 2023-07-06T02:23:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.519+0000 7fa3d77fe700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3dc06ace0 0x7fa3dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.519+0000 7fa3d77fe700 1 -- 172.21.15.89:0/1004674908 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa3e0089d80 con 0x7fa3f0151290 2023-07-06T02:23:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.519+0000 7fa3eeffd700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3dc06ace0 0x7fa3dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.519+0000 7fa3eeffd700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3dc06ace0 0x7fa3dc06d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4007fd0 tx=0x7fa3e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.523+0000 7fa3d77fe700 1 -- 172.21.15.89:0/1004674908 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3e00585d0 con 0x7fa3f0151290 2023-07-06T02:23:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.713+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3f0074270 con 0x7fa3f0151290 2023-07-06T02:23:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.713+0000 7fa3d77fe700 1 -- 172.21.15.89:0/1004674908 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3e0019020 con 0x7fa3f0151290 2023-07-06T02:23:04.715 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3dc06ace0 msgr2=0x7fa3dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3dc06ace0 0x7fa3dc06d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4007fd0 tx=0x7fa3e4009040 comp rx=0 tx=0).stop 2023-07-06T02:23:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f0151290 msgr2=0x7fa3f00c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f0151290 0x7fa3f00c8150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00119c0 tx=0x7fa3e0011d80 comp rx=0 tx=0).stop 2023-07-06T02:23:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 shutdown_connections 2023-07-06T02:23:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f00c8690 0x7fa3f00b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d8004d90 0x7fa3f0150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3dc06ace0 0x7fa3dc06d190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 --2- 172.21.15.89:0/1004674908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f0151290 0x7fa3f00c8150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.715+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 >> 172.21.15.89:0/1004674908 conn(0x7fa3f00cd0f0 msgr2=0x7fa3f006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.716+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 shutdown_connections 2023-07-06T02:23:04.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.716+0000 7fa3f56f6700 1 -- 172.21.15.89:0/1004674908 wait complete. 2023-07-06T02:23:04.729 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:04.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.850+0000 7fc6a49f3700 1 Processor -- start 2023-07-06T02:23:04.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.850+0000 7fc6a49f3700 1 -- start start 2023-07-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc6a49f3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a0156ab0 0x7fc6a0160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc6a49f3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6a0160bb0 0x7fc6a00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc6a49f3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a00cb3d0 con 0x7fc6a01600b0 2023-07-06T02:23:04.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc6a49f3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a0156e80 con 0x7fc6a0160bb0 2023-07-06T02:23:04.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc6a49f3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00ca370 con 0x7fc6a0156ab0 2023-07-06T02:23:04.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc69dd9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc6a0160bb0 0x7fc6a00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc69ed9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc6a01600b0 0x7fc6a006d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57130/0 (socket says 172.21.15.89:57130) 2023-07-06T02:23:04.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.851+0000 7fc69ed9d700 1 -- 172.21.15.89:0/780240662 learned_addr learned my addr 172.21.15.89:0/780240662 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:04.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69e59c700 1 --2- 172.21.15.89:0/780240662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a0156ab0 0x7fc6a0160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69dd9b700 1 -- 172.21.15.89:0/780240662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a0156ab0 msgr2=0x7fc6a0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69dd9b700 1 --2- 172.21.15.89:0/780240662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a0156ab0 0x7fc6a0160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69dd9b700 1 -- 172.21.15.89:0/780240662 >> v1:172.21.15.89:6789/0 conn(0x7fc6a01600b0 legacy=0x7fc6a006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69d59a700 1 -- 172.21.15.89:0/780240662 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026003640 0 0) 0x7fc6a00cb3d0 con 0x7fc6a01600b0 2023-07-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69dd9b700 1 -- 172.21.15.89:0/780240662 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00ca610 con 0x7fc6a0160bb0 2023-07-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69dd9b700 1 --2- 172.21.15.89:0/780240662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0160bb0 0x7fc6a00b1490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc690009840 tx=0x7fc690009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610d8009766d4965 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69d59a700 1 -- 172.21.15.89:0/780240662 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690011830 con 0x7fc6a0160bb0 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69d59a700 1 -- 172.21.15.89:0/780240662 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc690011990 con 0x7fc6a0160bb0 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc69d59a700 1 -- 172.21.15.89:0/780240662 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690011d20 con 0x7fc6a0160bb0 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc6a49f3700 1 -- 172.21.15.89:0/780240662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0160bb0 msgr2=0x7fc6a00b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.852+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/780240662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0160bb0 0x7fc6a00b1490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc690009840 tx=0x7fc690009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.853+0000 7fc6a49f3700 1 -- 172.21.15.89:0/780240662 shutdown_connections 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.853+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/780240662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a0156ab0 0x7fc6a0160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.853+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/780240662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0160bb0 0x7fc6a00b1490 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.853+0000 7fc6a49f3700 1 -- 172.21.15.89:0/780240662 >> 172.21.15.89:0/780240662 conn(0x7fc6a00cc490 msgr2=0x7fc6a00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.853+0000 7fc6a49f3700 1 -- 172.21.15.89:0/780240662 shutdown_connections 2023-07-06T02:23:04.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.853+0000 7fc6a49f3700 1 -- 172.21.15.89:0/780240662 wait complete. 2023-07-06T02:23:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.854+0000 7fc6a49f3700 1 Processor -- start 2023-07-06T02:23:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.854+0000 7fc6a49f3700 1 -- start start 2023-07-06T02:23:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc6a49f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0156ab0 0x7fc6a00b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc6a49f3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a01600b0 0x7fc6a00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc6a49f3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a00b0e00 0x7fc6a00c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc6a49f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00afef0 con 0x7fc6a0156ab0 2023-07-06T02:23:04.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc6a49f3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00afcc0 con 0x7fc6a01600b0 2023-07-06T02:23:04.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc6a49f3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00c9820 con 0x7fc6a00b0e00 2023-07-06T02:23:04.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0156ab0 0x7fc6a00b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0156ab0 0x7fc6a00b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60462/0 (socket says 172.21.15.89:60462) 2023-07-06T02:23:04.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 -- 172.21.15.89:0/2612325584 learned_addr learned my addr 172.21.15.89:0/2612325584 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:04.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 -- 172.21.15.89:0/2612325584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a00b0e00 msgr2=0x7fc6a00c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a00b0e00 0x7fc6a00c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 -- 172.21.15.89:0/2612325584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a01600b0 msgr2=0x7fc6a00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a01600b0 0x7fc6a00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 -- 172.21.15.89:0/2612325584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6900094a0 con 0x7fc6a0156ab0 2023-07-06T02:23:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc69e59c700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0156ab0 0x7fc6a00b0380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc69400e810 tx=0x7fc69400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc68effd700 1 -- 172.21.15.89:0/2612325584 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694013610 con 0x7fc6a0156ab0 2023-07-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc68effd700 1 -- 172.21.15.89:0/2612325584 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc694013770 con 0x7fc6a0156ab0 2023-07-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a01e87c0 con 0x7fc6a0156ab0 2023-07-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.855+0000 7fc68effd700 1 -- 172.21.15.89:0/2612325584 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694013b20 con 0x7fc6a0156ab0 2023-07-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.856+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a01e84b0 con 0x7fc6a0156ab0 2023-07-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.856+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a0074270 con 0x7fc6a0156ab0 2023-07-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.857+0000 7fc68effd700 1 -- 172.21.15.89:0/2612325584 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc694013c80 con 0x7fc6a0156ab0 2023-07-06T02:23:04.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.857+0000 7fc68effd700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc68006ad10 0x7fc68006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.858+0000 7fc69dd9b700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc68006ad10 0x7fc68006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.858+0000 7fc68effd700 1 -- 172.21.15.89:0/2612325584 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc6940895a0 con 0x7fc6a0156ab0 2023-07-06T02:23:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.858+0000 7fc69dd9b700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc68006ad10 0x7fc68006d1c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00c7fe0 tx=0x7fc690010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:04.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:04.862+0000 7fc68effd700 1 -- 172.21.15.89:0/2612325584 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc694054240 con 0x7fc6a0156ab0 2023-07-06T02:23:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.047+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6a00b5990 con 0x7fc6a0156ab0 2023-07-06T02:23:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.048+0000 7fc68effd700 1 -- 172.21.15.89:0/2612325584 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc694057860 con 0x7fc6a0156ab0 2023-07-06T02:23:05.050 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:05.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc68006ad10 msgr2=0x7fc68006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc68006ad10 0x7fc68006d1c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00c7fe0 tx=0x7fc690010000 comp rx=0 tx=0).stop 2023-07-06T02:23:05.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0156ab0 msgr2=0x7fc6a00b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0156ab0 0x7fc6a00b0380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc69400e810 tx=0x7fc69400ebd0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 shutdown_connections 2023-07-06T02:23:05.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6a00b0e00 0x7fc6a00c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc68006ad10 0x7fc68006d1c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6a0156ab0 0x7fc6a00b0380 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 --2- 172.21.15.89:0/2612325584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a01600b0 0x7fc6a00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 >> 172.21.15.89:0/2612325584 conn(0x7fc6a00cc490 msgr2=0x7fc6a00cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:05.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 shutdown_connections 2023-07-06T02:23:05.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.050+0000 7fc6a49f3700 1 -- 172.21.15.89:0/2612325584 wait complete. 2023-07-06T02:23:05.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:05.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.184+0000 7f6e088db700 1 Processor -- start 2023-07-06T02:23:05.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.184+0000 7f6e088db700 1 -- start start 2023-07-06T02:23:05.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e088db700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 0x7f6e040ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e088db700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0406b160 con 0x7f6e040cecf0 2023-07-06T02:23:05.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e088db700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e040cac10 con 0x7f6e040c9ee0 2023-07-06T02:23:05.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e088db700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040caa30 con 0x7f6e0406e960 2023-07-06T02:23:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e0259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6e040cecf0 0x7f6e0406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57140/0 (socket says 172.21.15.89:57140) 2023-07-06T02:23:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e0259c700 1 -- 172.21.15.89:0/3083931883 learned_addr learned my addr 172.21.15.89:0/3083931883 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e01d9b700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 0x7f6e040ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932562824 0 0) 0x7f6e0406b160 con 0x7f6e040cecf0 2023-07-06T02:23:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dec0035b0 con 0x7f6e040cecf0 2023-07-06T02:23:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.185+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3777608601 0 0) 0x7f6dec0035b0 con 0x7f6e040cecf0 2023-07-06T02:23:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 msgr2=0x7f6e040ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 0x7f6e040ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 >> v1:172.21.15.172:6789/0 conn(0x7f6e040c9ee0 legacy=0x7f6e040ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e040caef0 con 0x7f6e040cecf0 2023-07-06T02:23:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f6df8001670 con 0x7f6e040cecf0 2023-07-06T02:23:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 >> v1:172.21.15.89:6789/0 conn(0x7f6e040cecf0 legacy=0x7f6e0406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6dec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6dec0043d0 0x7f6dec004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 0x7f6dec0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040caa30 con 0x7f6dec003a30 2023-07-06T02:23:05.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec00ba10 con 0x7f6dec0043d0 2023-07-06T02:23:05.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec00ddb0 con 0x7f6e0406e960 2023-07-06T02:23:05.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0259c700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6dec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e01d9b700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6dec0043d0 0x7f6dec004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e02d9d700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 0x7f6dec0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0259c700 1 -- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 msgr2=0x7f6dec0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0259c700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 0x7f6dec0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0259c700 1 -- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6dec0043d0 msgr2=0x7f6dec004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0259c700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6dec0043d0 0x7f6dec004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0259c700 1 -- 172.21.15.89:0/3083931883 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0035b0 con 0x7f6dec003a30 2023-07-06T02:23:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e02d9d700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e0406e960 0x7f6dec0050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0259c700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6dec003e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c2f0 tx=0x7f6dec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9622d9dc3b9134c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800ce30 con 0x7f6dec003a30 2023-07-06T02:23:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df8005960 con 0x7f6dec003a30 2023-07-06T02:23:05.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.186+0000 7f6e0159a700 1 -- 172.21.15.89:0/3083931883 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8005cf0 con 0x7f6dec003a30 2023-07-06T02:23:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.187+0000 7f6e088db700 1 -- 172.21.15.89:0/3083931883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 msgr2=0x7f6dec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.187+0000 7f6e088db700 1 --2- 172.21.15.89:0/3083931883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6dec003e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c2f0 tx=0x7f6dec005880 comp rx=0 tx=0).stop 2023-07-06T02:23:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.187+0000 7f6e02d9d700 1 -- 172.21.15.89:0/3083931883 reap_dead start 2023-07-06T02:23:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.187+0000 7f6e088db700 1 -- 172.21.15.89:0/3083931883 shutdown_connections 2023-07-06T02:23:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.187+0000 7f6e088db700 1 -- 172.21.15.89:0/3083931883 >> 172.21.15.89:0/3083931883 conn(0x7f6e040cf1c0 msgr2=0x7f6e040cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:05.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.187+0000 7f6e088db700 1 -- 172.21.15.89:0/3083931883 shutdown_connections 2023-07-06T02:23:05.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.187+0000 7f6e088db700 1 -- 172.21.15.89:0/3083931883 wait complete. 2023-07-06T02:23:05.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.188+0000 7f6e088db700 1 Processor -- start 2023-07-06T02:23:05.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.188+0000 7f6e088db700 1 -- start start 2023-07-06T02:23:05.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e088db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6e040c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e088db700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c4c80 0x7f6e0414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e088db700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04071190 0x7f6e04071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e088db700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040aeb20 con 0x7f6dec003a30 2023-07-06T02:23:05.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e088db700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04071b40 con 0x7f6e040c4c80 2023-07-06T02:23:05.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e088db700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040c50f0 con 0x7f6e04071190 2023-07-06T02:23:05.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6e040c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6e040c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60482/0 (socket says 172.21.15.89:60482) 2023-07-06T02:23:05.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 -- 172.21.15.89:0/2274892197 learned_addr learned my addr 172.21.15.89:0/2274892197 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:05.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e01d9b700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c4c80 0x7f6e0414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e02d9d700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04071190 0x7f6e04071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 -- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04071190 msgr2=0x7f6e04071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04071190 0x7f6e04071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 -- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c4c80 msgr2=0x7f6e0414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c4c80 0x7f6e0414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.189+0000 7f6e0259c700 1 -- 172.21.15.89:0/2274892197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e040aed00 con 0x7f6dec003a30 2023-07-06T02:23:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.190+0000 7f6e0259c700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6e040c3c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005800 tx=0x7f6df8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.190+0000 7f6df2ffd700 1 -- 172.21.15.89:0/2274892197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8001660 con 0x7f6dec003a30 2023-07-06T02:23:05.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.190+0000 7f6df2ffd700 1 -- 172.21.15.89:0/2274892197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df8004a80 con 0x7f6dec003a30 2023-07-06T02:23:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.190+0000 7f6df2ffd700 1 -- 172.21.15.89:0/2274892197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8004e30 con 0x7f6dec003a30 2023-07-06T02:23:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.190+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e040aefe0 con 0x7f6dec003a30 2023-07-06T02:23:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.190+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e040af5a0 con 0x7f6dec003a30 2023-07-06T02:23:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.191+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e04074270 con 0x7f6dec003a30 2023-07-06T02:23:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.191+0000 7f6df2ffd700 1 -- 172.21.15.89:0/2274892197 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6df80070d0 con 0x7f6dec003a30 2023-07-06T02:23:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.191+0000 7f6df2ffd700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6de406ace0 0x7f6de406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.192+0000 7f6df2ffd700 1 -- 172.21.15.89:0/2274892197 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6df808aea0 con 0x7f6dec003a30 2023-07-06T02:23:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.192+0000 7f6e01d9b700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6de406ace0 0x7f6de406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.192+0000 7f6e01d9b700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6de406ace0 0x7f6de406d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e04153420 tx=0x7f6df4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.196+0000 7f6df2ffd700 1 -- 172.21.15.89:0/2274892197 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6df8055cf0 con 0x7f6dec003a30 2023-07-06T02:23:05.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.382+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6e040cd1d0 con 0x7f6dec003a30 2023-07-06T02:23:05.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.382+0000 7f6df2ffd700 1 -- 172.21.15.89:0/2274892197 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6df8022020 con 0x7f6dec003a30 2023-07-06T02:23:05.384 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:05.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6de406ace0 msgr2=0x7f6de406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6de406ace0 0x7f6de406d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e04153420 tx=0x7f6df4008040 comp rx=0 tx=0).stop 2023-07-06T02:23:05.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 msgr2=0x7f6e040c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6e040c3c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005800 tx=0x7f6df8001a00 comp rx=0 tx=0).stop 2023-07-06T02:23:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 shutdown_connections 2023-07-06T02:23:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04071190 0x7f6e04071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6de406ace0 0x7f6de406d190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dec003a30 0x7f6e040c3c90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 --2- 172.21.15.89:0/2274892197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c4c80 0x7f6e0414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 >> 172.21.15.89:0/2274892197 conn(0x7f6e040cf1c0 msgr2=0x7f6e04158210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:05.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 shutdown_connections 2023-07-06T02:23:05.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.384+0000 7f6e088db700 1 -- 172.21.15.89:0/2274892197 wait complete. 2023-07-06T02:23:05.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T02:23:05.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.522+0000 7f990e08a700 1 Processor -- start 2023-07-06T02:23:05.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.522+0000 7f990e08a700 1 -- start start 2023-07-06T02:23:05.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.522+0000 7f990e08a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.522+0000 7f990e08a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99080b19d0 0x7f99080cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.522+0000 7f990e08a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99080cb3d0 con 0x7f9908156ab0 2023-07-06T02:23:05.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.522+0000 7f990e08a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99080b1df0 con 0x7f99080b19d0 2023-07-06T02:23:05.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.522+0000 7f990e08a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9908156e80 con 0x7f99081600b0 2023-07-06T02:23:05.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99080b19d0 0x7f99080cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99080b19d0 0x7f99080cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60498/0 (socket says 172.21.15.89:60498) 2023-07-06T02:23:05.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 -- 172.21.15.89:0/3642191163 learned_addr learned my addr 172.21.15.89:0/3642191163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:05.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f99077fe700 1 --1- 172.21.15.89:0/3642191163 >> v1:172.21.15.89:6789/0 conn(0x7f9908156ab0 0x7f9908160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57164/0 (socket says 172.21.15.89:57164) 2023-07-06T02:23:05.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9906ffd700 1 --2- 172.21.15.89:0/3642191163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 -- 172.21.15.89:0/3642191163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 msgr2=0x7f99080b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:05.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 --2- 172.21.15.89:0/3642191163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 -- 172.21.15.89:0/3642191163 >> v1:172.21.15.89:6789/0 conn(0x7f9908156ab0 legacy=0x7f9908160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 -- 172.21.15.89:0/3642191163 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99080ca370 con 0x7f99080b19d0 2023-07-06T02:23:05.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f99067fc700 1 -- 172.21.15.89:0/3642191163 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185439955 0 0) 0x7f99080cb3d0 con 0x7f9908156ab0 2023-07-06T02:23:05.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f9907fff700 1 --2- 172.21.15.89:0/3642191163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99080b19d0 0x7f99080cd700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9900009e50 tx=0x7f990000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc76e70df2d329e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f99067fc700 1 -- 172.21.15.89:0/3642191163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000e070 con 0x7f99080b19d0 2023-07-06T02:23:05.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f99067fc700 1 -- 172.21.15.89:0/3642191163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990000b9c0 con 0x7f99080b19d0 2023-07-06T02:23:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.523+0000 7f99067fc700 1 -- 172.21.15.89:0/3642191163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000bd70 con 0x7f99080b19d0 2023-07-06T02:23:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 -- 172.21.15.89:0/3642191163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99080b19d0 msgr2=0x7f99080cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 --2- 172.21.15.89:0/3642191163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99080b19d0 0x7f99080cd700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9900009e50 tx=0x7f990000b330 comp rx=0 tx=0).stop 2023-07-06T02:23:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 -- 172.21.15.89:0/3642191163 shutdown_connections 2023-07-06T02:23:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 --2- 172.21.15.89:0/3642191163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99080b19d0 0x7f99080cd700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 --2- 172.21.15.89:0/3642191163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 -- 172.21.15.89:0/3642191163 >> 172.21.15.89:0/3642191163 conn(0x7f99080cc490 msgr2=0x7f99080cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:05.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 -- 172.21.15.89:0/3642191163 shutdown_connections 2023-07-06T02:23:05.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.524+0000 7f990e08a700 1 -- 172.21.15.89:0/3642191163 wait complete. 2023-07-06T02:23:05.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.525+0000 7f990e08a700 1 Processor -- start 2023-07-06T02:23:05.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.525+0000 7f990e08a700 1 -- start start 2023-07-06T02:23:05.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f990e08a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99080b19d0 0x7f99080c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f990e08a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9908156ab0 0x7f9908150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f990e08a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f990e08a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080af680 con 0x7f9908156ab0 2023-07-06T02:23:05.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f990e08a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080af330 con 0x7f99080b19d0 2023-07-06T02:23:05.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f990e08a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9908150580 con 0x7f99081600b0 2023-07-06T02:23:05.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9908156ab0 0x7f9908150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9908156ab0 0x7f9908150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60502/0 (socket says 172.21.15.89:60502) 2023-07-06T02:23:05.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 -- 172.21.15.89:0/2560007738 learned_addr learned my addr 172.21.15.89:0/2560007738 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:05.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f99077fe700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99080b19d0 0x7f99080c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9907fff700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 -- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 msgr2=0x7f99080c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 -- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99080b19d0 msgr2=0x7f99080c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99080b19d0 0x7f99080c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 -- 172.21.15.89:0/2560007738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9900009b30 con 0x7f9908156ab0 2023-07-06T02:23:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f9906ffd700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9908156ab0 0x7f9908150040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f98f800d830 tx=0x7f98f800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f98fffff700 1 -- 172.21.15.89:0/2560007738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80125f0 con 0x7f9908156ab0 2023-07-06T02:23:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f98fffff700 1 -- 172.21.15.89:0/2560007738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f8012750 con 0x7f9908156ab0 2023-07-06T02:23:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99080c9060 con 0x7f9908156ab0 2023-07-06T02:23:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.526+0000 7f98fffff700 1 -- 172.21.15.89:0/2560007738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8012b00 con 0x7f9908156ab0 2023-07-06T02:23:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.527+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99080c9620 con 0x7f9908156ab0 2023-07-06T02:23:05.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.527+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99080c97e0 con 0x7f9908156ab0 2023-07-06T02:23:05.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.528+0000 7f98fffff700 1 -- 172.21.15.89:0/2560007738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98f8012c60 con 0x7f9908156ab0 2023-07-06T02:23:05.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.528+0000 7f98fffff700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f98e806ace0 0x7f98e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.528+0000 7f98fffff700 1 -- 172.21.15.89:0/2560007738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f98f8089a90 con 0x7f9908156ab0 2023-07-06T02:23:05.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.528+0000 7f99077fe700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f98e806ace0 0x7f98e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.529+0000 7f99077fe700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f98e806ace0 0x7f98e806d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000f60 tx=0x7f98f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.533+0000 7f98fffff700 1 -- 172.21.15.89:0/2560007738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f98f80548e0 con 0x7f9908156ab0 2023-07-06T02:23:05.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.737+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9908074270 con 0x7f9908156ab0 2023-07-06T02:23:05.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.738+0000 7f98fffff700 1 -- 172.21.15.89:0/2560007738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f98f8057f00 con 0x7f9908156ab0 2023-07-06T02:23:05.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f98e806ace0 msgr2=0x7f98e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f98e806ace0 0x7f98e806d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000f60 tx=0x7f98f0009040 comp rx=0 tx=0).stop 2023-07-06T02:23:05.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9908156ab0 msgr2=0x7f9908150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9908156ab0 0x7f9908150040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f98f800d830 tx=0x7f98f800dbf0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 shutdown_connections 2023-07-06T02:23:05.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99081600b0 0x7f99080c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99080b19d0 0x7f99080c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f98e806ace0 0x7f98e806d190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 --2- 172.21.15.89:0/2560007738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9908156ab0 0x7f9908150040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 >> 172.21.15.89:0/2560007738 conn(0x7f99080cc490 msgr2=0x7f990815f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:05.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 shutdown_connections 2023-07-06T02:23:05.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.740+0000 7f990e08a700 1 -- 172.21.15.89:0/2560007738 wait complete. 2023-07-06T02:23:05.746 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-07-06T02:23:05.753 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:05.754 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:05.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:05.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.928+0000 7f1f94936700 1 Processor -- start 2023-07-06T02:23:05.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.928+0000 7f1f94936700 1 -- start start 2023-07-06T02:23:05.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f94936700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1f900b1490 0x7f1f900b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f94936700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f9006bfa0 0x7f1f9006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f94936700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900cab50 con 0x7f1f9015c690 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f94936700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9015de50 con 0x7f1f9006bfa0 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f94936700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900b1e50 con 0x7f1f900b1490 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f9006bfa0 0x7f1f9006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8ed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f9006bfa0 0x7f1f9006c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60512/0 (socket says 172.21.15.89:60512) 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8ed9d700 1 -- 172.21.15.89:0/3676132546 learned_addr learned my addr 172.21.15.89:0/3676132546 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8dd9b700 1 --2- 172.21.15.89:0/3676132546 >> v2:172.21.15.172:3300/0 conn(0x7f1f900b1490 0x7f1f900b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8ed9d700 1 -- 172.21.15.89:0/3676132546 >> v2:172.21.15.172:3300/0 conn(0x7f1f900b1490 msgr2=0x7f1f900b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:05.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8ed9d700 1 --2- 172.21.15.89:0/3676132546 >> v2:172.21.15.172:3300/0 conn(0x7f1f900b1490 0x7f1f900b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8ed9d700 1 -- 172.21.15.89:0/3676132546 >> v1:172.21.15.172:6789/0 conn(0x7f1f9015c690 legacy=0x7f1f900cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8ed9d700 1 -- 172.21.15.89:0/3676132546 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9006c8e0 con 0x7f1f9006bfa0 2023-07-06T02:23:05.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.929+0000 7f1f8dd9b700 1 --2- 172.21.15.89:0/3676132546 >> v2:172.21.15.172:3300/0 conn(0x7f1f900b1490 0x7f1f900b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:05.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f8ed9d700 1 --2- 172.21.15.89:0/3676132546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9006bfa0 0x7f1f9006c3a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f90070180 tx=0x7f1f8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8875a2e7245fa34a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f8d59a700 1 -- 172.21.15.89:0/3676132546 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800e070 con 0x7f1f9006bfa0 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f8d59a700 1 -- 172.21.15.89:0/3676132546 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800b9c0 con 0x7f1f9006bfa0 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f8d59a700 1 -- 172.21.15.89:0/3676132546 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800bd70 con 0x7f1f9006bfa0 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 -- 172.21.15.89:0/3676132546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9006bfa0 msgr2=0x7f1f9006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 --2- 172.21.15.89:0/3676132546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9006bfa0 0x7f1f9006c3a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f90070180 tx=0x7f1f8800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 -- 172.21.15.89:0/3676132546 shutdown_connections 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 --2- 172.21.15.89:0/3676132546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9006bfa0 0x7f1f9006c3a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 --2- 172.21.15.89:0/3676132546 >> v2:172.21.15.172:3300/0 conn(0x7f1f900b1490 0x7f1f900b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 -- 172.21.15.89:0/3676132546 >> 172.21.15.89:0/3676132546 conn(0x7f1f900cc490 msgr2=0x7f1f900cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:05.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 -- 172.21.15.89:0/3676132546 shutdown_connections 2023-07-06T02:23:05.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.930+0000 7f1f94936700 1 -- 172.21.15.89:0/3676132546 wait complete. 2023-07-06T02:23:05.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 Processor -- start 2023-07-06T02:23:05.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 -- start start 2023-07-06T02:23:05.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f9006bfa0 0x7f1f900ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b1490 0x7f1f9014bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f9015c690 0x7f1f9014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90149b70 con 0x7f1f900b1490 2023-07-06T02:23:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90149e50 con 0x7f1f9015c690 2023-07-06T02:23:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.932+0000 7f1f94936700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9014a130 con 0x7f1f9006bfa0 2023-07-06T02:23:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b1490 0x7f1f9014bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b1490 0x7f1f9014bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60516/0 (socket says 172.21.15.89:60516) 2023-07-06T02:23:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 -- 172.21.15.89:0/3503074521 learned_addr learned my addr 172.21.15.89:0/3503074521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8ed9d700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f9015c690 0x7f1f9014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8e59c700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f9006bfa0 0x7f1f900ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 -- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f9006bfa0 msgr2=0x7f1f900ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f9006bfa0 0x7f1f900ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 -- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f9015c690 msgr2=0x7f1f9014c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f9015c690 0x7f1f9014c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 -- 172.21.15.89:0/3503074521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88009b30 con 0x7f1f900b1490 2023-07-06T02:23:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f8dd9b700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b1490 0x7f1f9014bad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e810 tx=0x7f1f8000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f7effd700 1 -- 172.21.15.89:0/3503074521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80013610 con 0x7f1f900b1490 2023-07-06T02:23:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f7effd700 1 -- 172.21.15.89:0/3503074521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80013770 con 0x7f1f900b1490 2023-07-06T02:23:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9014a410 con 0x7f1f900b1490 2023-07-06T02:23:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f7effd700 1 -- 172.21.15.89:0/3503074521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80013b20 con 0x7f1f900b1490 2023-07-06T02:23:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.933+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9014c790 con 0x7f1f900b1490 2023-07-06T02:23:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.934+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f90074270 con 0x7f1f900b1490 2023-07-06T02:23:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.934+0000 7f1f7effd700 1 -- 172.21.15.89:0/3503074521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f80013c80 con 0x7f1f900b1490 2023-07-06T02:23:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.935+0000 7f1f7effd700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f7006ace0 0x7f1f7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.935+0000 7f1f8e59c700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f7006ace0 0x7f1f7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:05.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.935+0000 7f1f7effd700 1 -- 172.21.15.89:0/3503074521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f80089240 con 0x7f1f900b1490 2023-07-06T02:23:05.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.935+0000 7f1f8e59c700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f7006ace0 0x7f1f7006d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007fd0 tx=0x7f1f84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:05.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:05.939+0000 7f1f7effd700 1 -- 172.21.15.89:0/3503074521 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1f80057b20 con 0x7f1f900b1490 2023-07-06T02:23:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.128+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f9006bd80 con 0x7f1f900b1490 2023-07-06T02:23:06.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.129+0000 7f1f7effd700 1 -- 172.21.15.89:0/3503074521 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f1f800576b0 con 0x7f1f900b1490 2023-07-06T02:23:06.131 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:06.132 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:06.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f7006ace0 msgr2=0x7f1f7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f7006ace0 0x7f1f7006d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007fd0 tx=0x7f1f84009040 comp rx=0 tx=0).stop 2023-07-06T02:23:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b1490 msgr2=0x7f1f9014bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b1490 0x7f1f9014bad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e810 tx=0x7f1f8000eb20 comp rx=0 tx=0).stop 2023-07-06T02:23:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 shutdown_connections 2023-07-06T02:23:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f9015c690 0x7f1f9014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f9006bfa0 0x7f1f900ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f7006ace0 0x7f1f7006d190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 --2- 172.21.15.89:0/3503074521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b1490 0x7f1f9014bad0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 >> 172.21.15.89:0/3503074521 conn(0x7f1f900cc490 msgr2=0x7f1f900d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 shutdown_connections 2023-07-06T02:23:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.132+0000 7f1f94936700 1 -- 172.21.15.89:0/3503074521 wait complete. 2023-07-06T02:23:06.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:06.147 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.269+0000 7f2dc94ee700 1 Processor -- start 2023-07-06T02:23:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.269+0000 7f2dc94ee700 1 -- start start 2023-07-06T02:23:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc94ee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2dc40b8480 0x7f2dc40cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc94ee700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2dc4160de0 0x7f2dc40b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc94ee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b51a0 0x7f2dc406bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc94ee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc406b7d0 con 0x7f2dc40b8480 2023-07-06T02:23:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc94ee700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc406afc0 con 0x7f2dc4160de0 2023-07-06T02:23:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc94ee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40b55c0 con 0x7f2dc40b51a0 2023-07-06T02:23:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2dc40b8480 0x7f2dc40cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2dc40b8480 0x7f2dc40cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60522/0 (socket says 172.21.15.89:60522) 2023-07-06T02:23:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 -- 172.21.15.89:0/1669036092 learned_addr learned my addr 172.21.15.89:0/1669036092 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc27fc700 1 --2- 172.21.15.89:0/1669036092 >> v2:172.21.15.172:3300/0 conn(0x7f2dc4160de0 0x7f2dc40b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc37fe700 1 --2- 172.21.15.89:0/1669036092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b51a0 0x7f2dc406bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 -- 172.21.15.89:0/1669036092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b51a0 msgr2=0x7f2dc406bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 --2- 172.21.15.89:0/1669036092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b51a0 0x7f2dc406bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 -- 172.21.15.89:0/1669036092 >> v2:172.21.15.172:3300/0 conn(0x7f2dc4160de0 msgr2=0x7f2dc40b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 --2- 172.21.15.89:0/1669036092 >> v2:172.21.15.172:3300/0 conn(0x7f2dc4160de0 0x7f2dc40b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 -- 172.21.15.89:0/1669036092 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc406b210 con 0x7f2dc40b8480 2023-07-06T02:23:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc37fe700 1 --2- 172.21.15.89:0/1669036092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b51a0 0x7f2dc406bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc27fc700 1 --2- 172.21.15.89:0/1669036092 >> v2:172.21.15.172:3300/0 conn(0x7f2dc4160de0 0x7f2dc40b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc2ffd700 1 --2- 172.21.15.89:0/1669036092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc40b8480 0x7f2dc40cd320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40b5880 tx=0x7f2db8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa3ac3774708b85 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc1ffb700 1 -- 172.21.15.89:0/1669036092 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800b870 con 0x7f2dc40b8480 2023-07-06T02:23:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc1ffb700 1 -- 172.21.15.89:0/1669036092 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800b9d0 con 0x7f2dc40b8480 2023-07-06T02:23:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.270+0000 7f2dc1ffb700 1 -- 172.21.15.89:0/1669036092 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800bd60 con 0x7f2dc40b8480 2023-07-06T02:23:06.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 -- 172.21.15.89:0/1669036092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc40b8480 msgr2=0x7f2dc40cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/1669036092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc40b8480 0x7f2dc40cd320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40b5880 tx=0x7f2db8009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 -- 172.21.15.89:0/1669036092 shutdown_connections 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/1669036092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b51a0 0x7f2dc406bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/1669036092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc40b8480 0x7f2dc40cd320 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/1669036092 >> v2:172.21.15.172:3300/0 conn(0x7f2dc4160de0 0x7f2dc40b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 -- 172.21.15.89:0/1669036092 >> 172.21.15.89:0/1669036092 conn(0x7f2dc40d2d60 msgr2=0x7f2dc40d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 -- 172.21.15.89:0/1669036092 shutdown_connections 2023-07-06T02:23:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.271+0000 7f2dc94ee700 1 -- 172.21.15.89:0/1669036092 wait complete. 2023-07-06T02:23:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.273+0000 7f2dc94ee700 1 Processor -- start 2023-07-06T02:23:06.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.273+0000 7f2dc94ee700 1 -- start start 2023-07-06T02:23:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc94ee700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dc40b51a0 0x7f2dc414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc94ee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b8480 0x7f2dc40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc94ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc4160de0 0x7f2dc41f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc94ee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc41f6370 con 0x7f2dc4160de0 2023-07-06T02:23:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc94ee700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc41f6090 con 0x7f2dc40b51a0 2023-07-06T02:23:06.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc94ee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc414ccf0 con 0x7f2dc40b8480 2023-07-06T02:23:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc4160de0 0x7f2dc41f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc4160de0 0x7f2dc41f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60532/0 (socket says 172.21.15.89:60532) 2023-07-06T02:23:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 -- 172.21.15.89:0/896148767 learned_addr learned my addr 172.21.15.89:0/896148767 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc27fc700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b8480 0x7f2dc40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc2ffd700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dc40b51a0 0x7f2dc414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 -- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b8480 msgr2=0x7f2dc40b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b8480 0x7f2dc40b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 -- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dc40b51a0 msgr2=0x7f2dc414d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dc40b51a0 0x7f2dc414d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 -- 172.21.15.89:0/896148767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8009550 con 0x7f2dc4160de0 2023-07-06T02:23:06.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc37fe700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc4160de0 0x7f2dc41f5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00cd50 tx=0x7f2dbc00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dab7fe700 1 -- 172.21.15.89:0/896148767 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc00cea0 con 0x7f2dc4160de0 2023-07-06T02:23:06.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dab7fe700 1 -- 172.21.15.89:0/896148767 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc00f480 con 0x7f2dc4160de0 2023-07-06T02:23:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.274+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc414cfd0 con 0x7f2dc4160de0 2023-07-06T02:23:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.275+0000 7f2dab7fe700 1 -- 172.21.15.89:0/896148767 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc00f830 con 0x7f2dc4160de0 2023-07-06T02:23:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.275+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc40b7350 con 0x7f2dc4160de0 2023-07-06T02:23:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.275+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc414d4f0 con 0x7f2dc4160de0 2023-07-06T02:23:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.276+0000 7f2dab7fe700 1 -- 172.21.15.89:0/896148767 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2dbc00b040 con 0x7f2dc4160de0 2023-07-06T02:23:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.276+0000 7f2dab7fe700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2db006ace0 0x7f2db006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.276+0000 7f2dab7fe700 1 -- 172.21.15.89:0/896148767 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2dbc089da0 con 0x7f2dc4160de0 2023-07-06T02:23:06.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.276+0000 7f2dc2ffd700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2db006ace0 0x7f2db006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.277+0000 7f2dc2ffd700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2db006ace0 0x7f2db006d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2db8014040 tx=0x7f2db8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.280+0000 7f2dab7fe700 1 -- 172.21.15.89:0/896148767 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2dc414d4f0 con 0x7f2dc4160de0 2023-07-06T02:23:06.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.468+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2dc414d4f0 con 0x7f2dc4160de0 2023-07-06T02:23:06.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.468+0000 7f2dab7fe700 1 -- 172.21.15.89:0/896148767 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2dc414d4f0 con 0x7f2dc4160de0 2023-07-06T02:23:06.470 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2db006ace0 msgr2=0x7f2db006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2db006ace0 0x7f2db006d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2db8014040 tx=0x7f2db8008030 comp rx=0 tx=0).stop 2023-07-06T02:23:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc4160de0 msgr2=0x7f2dc41f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc4160de0 0x7f2dc41f5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00cd50 tx=0x7f2dbc00a560 comp rx=0 tx=0).stop 2023-07-06T02:23:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 shutdown_connections 2023-07-06T02:23:06.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2db006ace0 0x7f2db006d190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc4160de0 0x7f2dc41f5460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dc40b51a0 0x7f2dc414d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 --2- 172.21.15.89:0/896148767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dc40b8480 0x7f2dc40b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 >> 172.21.15.89:0/896148767 conn(0x7f2dc40d2d60 msgr2=0x7f2dc415c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:06.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 shutdown_connections 2023-07-06T02:23:06.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.470+0000 7f2dc94ee700 1 -- 172.21.15.89:0/896148767 wait complete. 2023-07-06T02:23:06.483 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:06.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.605+0000 7fabc563d700 1 Processor -- start 2023-07-06T02:23:06.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.605+0000 7fabc563d700 1 -- start start 2023-07-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabc563d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc006b610 0x7fabc006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabc563d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fabc00ceda0 0x7fabc015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabc563d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc00cd240 con 0x7fabc006bdb0 2023-07-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabc563d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc00cf280 con 0x7fabc00ceda0 2023-07-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabc563d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00cd490 con 0x7fabc006b610 2023-07-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbeffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fabc006bdb0 0x7fabc006c180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57206/0 (socket says 172.21.15.89:57206) 2023-07-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbeffd700 1 -- 172.21.15.89:0/2482534848 learned_addr learned my addr 172.21.15.89:0/2482534848 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:06.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/2482534848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc006b610 0x7fabc006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbf7fe700 1 --2- 172.21.15.89:0/2482534848 >> v2:172.21.15.172:3300/0 conn(0x7fabc00ceda0 0x7fabc015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbdffb700 1 -- 172.21.15.89:0/2482534848 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489424368 0 0) 0x7fabc00cd240 con 0x7fabc006bdb0 2023-07-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbdffb700 1 -- 172.21.15.89:0/2482534848 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba80035b0 con 0x7fabc006bdb0 2023-07-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbdffb700 1 -- 172.21.15.89:0/2482534848 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2297701520 0 0) 0x7faba80035b0 con 0x7fabc006bdb0 2023-07-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbe7fc700 1 -- 172.21.15.89:0/2482534848 >> v2:172.21.15.172:3300/0 conn(0x7fabc00ceda0 msgr2=0x7fabc015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/2482534848 >> v2:172.21.15.172:3300/0 conn(0x7fabc00ceda0 0x7fabc015c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbe7fc700 1 -- 172.21.15.89:0/2482534848 >> v1:172.21.15.89:6789/0 conn(0x7fabc006bdb0 legacy=0x7fabc006c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.606+0000 7fabbe7fc700 1 -- 172.21.15.89:0/2482534848 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00cd770 con 0x7fabc006b610 2023-07-06T02:23:06.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/2482534848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc006b610 0x7fabc006aa70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009840 tx=0x7fabb4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4939d1cdc1459dd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabbdffb700 1 -- 172.21.15.89:0/2482534848 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0001670 con 0x7fabc006b610 2023-07-06T02:23:06.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabbdffb700 1 -- 172.21.15.89:0/2482534848 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb4004030 con 0x7fabc006b610 2023-07-06T02:23:06.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabbdffb700 1 -- 172.21.15.89:0/2482534848 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40043a0 con 0x7fabc006b610 2023-07-06T02:23:06.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 -- 172.21.15.89:0/2482534848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc006b610 msgr2=0x7fabc006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 --2- 172.21.15.89:0/2482534848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc006b610 0x7fabc006aa70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009840 tx=0x7fabb4009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:06.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 -- 172.21.15.89:0/2482534848 shutdown_connections 2023-07-06T02:23:06.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 --2- 172.21.15.89:0/2482534848 >> v2:172.21.15.172:3300/0 conn(0x7fabc00ceda0 0x7fabc015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 --2- 172.21.15.89:0/2482534848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc006b610 0x7fabc006aa70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 -- 172.21.15.89:0/2482534848 >> 172.21.15.89:0/2482534848 conn(0x7fabc00b5390 msgr2=0x7fabc00d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:06.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 -- 172.21.15.89:0/2482534848 shutdown_connections 2023-07-06T02:23:06.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.607+0000 7fabc563d700 1 -- 172.21.15.89:0/2482534848 wait complete. 2023-07-06T02:23:06.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 Processor -- start 2023-07-06T02:23:06.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 -- start start 2023-07-06T02:23:06.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bdb0 0x7fabc00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc00ceda0 0x7fabc0151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc00c6700 0x7fabc00c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0152540 con 0x7fabc00c6700 2023-07-06T02:23:06.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00ccb20 con 0x7fabc006bdb0 2023-07-06T02:23:06.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabc563d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00c9de0 con 0x7fabc00ceda0 2023-07-06T02:23:06.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabbf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc00c6700 0x7fabc00c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabbf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc00c6700 0x7fabc00c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60546/0 (socket says 172.21.15.89:60546) 2023-07-06T02:23:06.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.609+0000 7fabbf7fe700 1 -- 172.21.15.89:0/2194972680 learned_addr learned my addr 172.21.15.89:0/2194972680 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:06.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbe7fc700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc00ceda0 0x7fabc0151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbeffd700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bdb0 0x7fabc00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbf7fe700 1 -- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc00ceda0 msgr2=0x7fabc0151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbf7fe700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc00ceda0 0x7fabc0151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbf7fe700 1 -- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bdb0 msgr2=0x7fabc00cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbf7fe700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bdb0 0x7fabc00cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbf7fe700 1 -- 172.21.15.89:0/2194972680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40094a0 con 0x7fabc00c6700 2023-07-06T02:23:06.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbeffd700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bdb0 0x7fabc00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabbf7fe700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc00c6700 0x7fabc00c81f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabb800efc0 tx=0x7fabb800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7faba77fe700 1 -- 172.21.15.89:0/2194972680 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800cb80 con 0x7fabc00c6700 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7faba77fe700 1 -- 172.21.15.89:0/2194972680 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb800f040 con 0x7fabc00c6700 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc0151a70 con 0x7fabc00c6700 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7faba77fe700 1 -- 172.21.15.89:0/2194972680 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8014480 con 0x7fabc00c6700 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.610+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc00c91a0 con 0x7fabc00c6700 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.611+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc0072ff0 con 0x7fabc00c6700 2023-07-06T02:23:06.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.611+0000 7faba77fe700 1 -- 172.21.15.89:0/2194972680 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabb80145e0 con 0x7fabc00c6700 2023-07-06T02:23:06.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.612+0000 7faba77fe700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fabac06ace0 0x7fabac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.612+0000 7faba77fe700 1 -- 172.21.15.89:0/2194972680 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fabb8089f90 con 0x7fabc00c6700 2023-07-06T02:23:06.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.612+0000 7fabbeffd700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fabac06ace0 0x7fabac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.612+0000 7fabbeffd700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fabac06ace0 0x7fabac06d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabb0001eb0 tx=0x7fabb0001d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.616+0000 7faba77fe700 1 -- 172.21.15.89:0/2194972680 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabb80bc310 con 0x7fabc00c6700 2023-07-06T02:23:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.803+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabc00c9fc0 con 0x7fabc00c6700 2023-07-06T02:23:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.803+0000 7faba77fe700 1 -- 172.21.15.89:0/2194972680 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabb8018020 con 0x7fabc00c6700 2023-07-06T02:23:06.805 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:06.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fabac06ace0 msgr2=0x7fabac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fabac06ace0 0x7fabac06d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabb0001eb0 tx=0x7fabb0001d00 comp rx=0 tx=0).stop 2023-07-06T02:23:06.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc00c6700 msgr2=0x7fabc00c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc00c6700 0x7fabc00c81f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabb800efc0 tx=0x7fabb800c0c0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 shutdown_connections 2023-07-06T02:23:06.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fabac06ace0 0x7fabac06d190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc00c6700 0x7fabc00c81f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bdb0 0x7fabc00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 --2- 172.21.15.89:0/2194972680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabc00ceda0 0x7fabc0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 >> 172.21.15.89:0/2194972680 conn(0x7fabc00b5390 msgr2=0x7fabc00b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 shutdown_connections 2023-07-06T02:23:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.805+0000 7fabc563d700 1 -- 172.21.15.89:0/2194972680 wait complete. 2023-07-06T02:23:06.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.939+0000 7feb3cdd0700 1 Processor -- start 2023-07-06T02:23:06.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.939+0000 7feb3cdd0700 1 -- start start 2023-07-06T02:23:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.940+0000 7feb3cdd0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3815c690 0x7feb380cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.940+0000 7feb3cdd0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb380b1ba0 0x7feb3806bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.940+0000 7feb3cdd0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb380cab50 con 0x7feb380cbda0 2023-07-06T02:23:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.940+0000 7feb3cdd0700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3815de50 con 0x7feb380b1ba0 2023-07-06T02:23:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.940+0000 7feb3cdd0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3815db70 con 0x7feb3815c690 2023-07-06T02:23:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.940+0000 7feb35d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7feb380cbda0 0x7feb380b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57220/0 (socket says 172.21.15.89:57220) 2023-07-06T02:23:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb35d9b700 1 -- 172.21.15.89:0/3536181626 learned_addr learned my addr 172.21.15.89:0/3536181626 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3659c700 1 --2- 172.21.15.89:0/3536181626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3815c690 0x7feb380cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb36d9d700 1 --2- 172.21.15.89:0/3536181626 >> v2:172.21.15.172:3300/0 conn(0x7feb380b1ba0 0x7feb3806bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3559a700 1 -- 172.21.15.89:0/3536181626 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179175202 0 0) 0x7feb380cab50 con 0x7feb380cbda0 2023-07-06T02:23:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3559a700 1 -- 172.21.15.89:0/3536181626 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb200035b0 con 0x7feb380cbda0 2023-07-06T02:23:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3659c700 1 -- 172.21.15.89:0/3536181626 >> v2:172.21.15.172:3300/0 conn(0x7feb380b1ba0 msgr2=0x7feb3806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3659c700 1 --2- 172.21.15.89:0/3536181626 >> v2:172.21.15.172:3300/0 conn(0x7feb380b1ba0 0x7feb3806bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3659c700 1 -- 172.21.15.89:0/3536181626 >> v1:172.21.15.89:6789/0 conn(0x7feb380cbda0 legacy=0x7feb380b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3659c700 1 -- 172.21.15.89:0/3536181626 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380cad80 con 0x7feb3815c690 2023-07-06T02:23:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3659c700 1 --2- 172.21.15.89:0/3536181626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3815c690 0x7feb380cb860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009840 tx=0x7feb2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811083e9c248eded server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3559a700 1 -- 172.21.15.89:0/3536181626 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c011830 con 0x7feb3815c690 2023-07-06T02:23:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3559a700 1 -- 172.21.15.89:0/3536181626 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c011990 con 0x7feb3815c690 2023-07-06T02:23:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.941+0000 7feb3559a700 1 -- 172.21.15.89:0/3536181626 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c011d20 con 0x7feb3815c690 2023-07-06T02:23:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 -- 172.21.15.89:0/3536181626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3815c690 msgr2=0x7feb380cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/3536181626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3815c690 0x7feb380cb860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009840 tx=0x7feb2c009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 -- 172.21.15.89:0/3536181626 shutdown_connections 2023-07-06T02:23:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/3536181626 >> v2:172.21.15.172:3300/0 conn(0x7feb380b1ba0 0x7feb3806bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/3536181626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3815c690 0x7feb380cb860 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 -- 172.21.15.89:0/3536181626 >> 172.21.15.89:0/3536181626 conn(0x7feb380cc490 msgr2=0x7feb380cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 -- 172.21.15.89:0/3536181626 shutdown_connections 2023-07-06T02:23:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.942+0000 7feb3cdd0700 1 -- 172.21.15.89:0/3536181626 wait complete. 2023-07-06T02:23:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.943+0000 7feb3cdd0700 1 Processor -- start 2023-07-06T02:23:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.943+0000 7feb3cdd0700 1 -- start start 2023-07-06T02:23:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3cdd0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb380b1ba0 0x7feb3814feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3cdd0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb380cbda0 0x7feb3806d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3cdd0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3815c690 0x7feb3806de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3cdd0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380c7bf0 con 0x7feb380b1ba0 2023-07-06T02:23:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3cdd0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380c7ed0 con 0x7feb3815c690 2023-07-06T02:23:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3cdd0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb380c81b0 con 0x7feb380cbda0 2023-07-06T02:23:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb380b1ba0 0x7feb3814feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb380b1ba0 0x7feb3814feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60556/0 (socket says 172.21.15.89:60556) 2023-07-06T02:23:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb3659c700 1 -- 172.21.15.89:0/1737408575 learned_addr learned my addr 172.21.15.89:0/1737408575 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb36d9d700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3815c690 0x7feb3806de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.944+0000 7feb35d9b700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb380cbda0 0x7feb3806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3659c700 1 -- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb380cbda0 msgr2=0x7feb3806d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3659c700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb380cbda0 0x7feb3806d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3659c700 1 -- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3815c690 msgr2=0x7feb3806de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3659c700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3815c690 0x7feb3806de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3659c700 1 -- 172.21.15.89:0/1737408575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c0094a0 con 0x7feb380b1ba0 2023-07-06T02:23:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3659c700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb380b1ba0 0x7feb3814feb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009b90 tx=0x7feb2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb26ffd700 1 -- 172.21.15.89:0/1737408575 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0036a0 con 0x7feb380b1ba0 2023-07-06T02:23:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb26ffd700 1 -- 172.21.15.89:0/1737408575 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c003800 con 0x7feb380b1ba0 2023-07-06T02:23:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb380c8490 con 0x7feb380b1ba0 2023-07-06T02:23:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb26ffd700 1 -- 172.21.15.89:0/1737408575 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c003b90 con 0x7feb380b1ba0 2023-07-06T02:23:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.945+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb381503f0 con 0x7feb380b1ba0 2023-07-06T02:23:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.946+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb380b2120 con 0x7feb380b1ba0 2023-07-06T02:23:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.946+0000 7feb26ffd700 1 -- 172.21.15.89:0/1737408575 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb2c003cf0 con 0x7feb380b1ba0 2023-07-06T02:23:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.947+0000 7feb26ffd700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feb1806ad10 0x7feb1806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.947+0000 7feb35d9b700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feb1806ad10 0x7feb1806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.947+0000 7feb26ffd700 1 -- 172.21.15.89:0/1737408575 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7feb2c08ab60 con 0x7feb380b1ba0 2023-07-06T02:23:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.947+0000 7feb35d9b700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feb1806ad10 0x7feb1806d1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb3806b930 tx=0x7feb28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:06.951+0000 7feb26ffd700 1 -- 172.21.15.89:0/1737408575 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb2c055ad0 con 0x7feb380b1ba0 2023-07-06T02:23:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.139+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb380c7d30 con 0x7feb380b1ba0 2023-07-06T02:23:07.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.139+0000 7feb26ffd700 1 -- 172.21.15.89:0/1737408575 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb2c015020 con 0x7feb380b1ba0 2023-07-06T02:23:07.141 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feb1806ad10 msgr2=0x7feb1806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feb1806ad10 0x7feb1806d1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb3806b930 tx=0x7feb28009040 comp rx=0 tx=0).stop 2023-07-06T02:23:07.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb380b1ba0 msgr2=0x7feb3814feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb380b1ba0 0x7feb3814feb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009b90 tx=0x7feb2c004750 comp rx=0 tx=0).stop 2023-07-06T02:23:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 shutdown_connections 2023-07-06T02:23:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3815c690 0x7feb3806de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feb1806ad10 0x7feb1806d1c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feb380b1ba0 0x7feb3814feb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 --2- 172.21.15.89:0/1737408575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb380cbda0 0x7feb3806d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 >> 172.21.15.89:0/1737408575 conn(0x7feb380cc490 msgr2=0x7feb3806af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 shutdown_connections 2023-07-06T02:23:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.141+0000 7feb3cdd0700 1 -- 172.21.15.89:0/1737408575 wait complete. 2023-07-06T02:23:07.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:07.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.276+0000 7f70ebc02700 1 Processor -- start 2023-07-06T02:23:07.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.277+0000 7f70ebc02700 1 -- start start 2023-07-06T02:23:07.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.277+0000 7f70ebc02700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 0x7f70e40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.277+0000 7f70ebc02700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e40cb3d0 con 0x7f70e40b19d0 2023-07-06T02:23:07.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.277+0000 7f70ebc02700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e40b1df0 con 0x7f70e4156ab0 2023-07-06T02:23:07.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.277+0000 7f70ebc02700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4156e80 con 0x7f70e41600b0 2023-07-06T02:23:07.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.277+0000 7f70ea19f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f70e40b19d0 0x7f70e40cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57238/0 (socket says 172.21.15.89:57238) 2023-07-06T02:23:07.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70ea19f700 1 -- 172.21.15.89:0/1555034826 learned_addr learned my addr 172.21.15.89:0/1555034826 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:07.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742760363 0 0) 0x7f70e40cb3d0 con 0x7f70e40b19d0 2023-07-06T02:23:07.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e919d700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 0x7f70e40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70d00035b0 con 0x7f70e40b19d0 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431453803 0 0) 0x7f70d00035b0 con 0x7f70e40b19d0 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 msgr2=0x7f70e40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 0x7f70e40b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 >> v1:172.21.15.172:6789/0 conn(0x7f70e4156ab0 legacy=0x7f70e4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40ca370 con 0x7f70e40b19d0 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f70dc0019f0 con 0x7f70e40b19d0 2023-07-06T02:23:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 >> v1:172.21.15.89:6789/0 conn(0x7f70e40b19d0 legacy=0x7f70e40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 0x7f70d0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d0004000 0x7f70d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4156e80 con 0x7f70d0004d90 2023-07-06T02:23:07.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d000bb20 con 0x7f70d0004000 2023-07-06T02:23:07.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d000bce0 con 0x7f70e41600b0 2023-07-06T02:23:07.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70ea19f700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e919d700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d0004000 0x7f70d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70e999e700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 0x7f70d0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70ea19f700 1 -- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 msgr2=0x7f70d0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70ea19f700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41600b0 0x7f70d0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70ea19f700 1 -- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d0004000 msgr2=0x7f70d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.278+0000 7f70ea19f700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d0004000 0x7f70d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ea19f700 1 -- 172.21.15.89:0/1555034826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00035b0 con 0x7f70d0004d90 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ea19f700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70d0005200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00a760 tx=0x7f70d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc3bad66f32934b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00a250 con 0x7f70d0004d90 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0072d0 con 0x7f70d0004d90 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70e899c700 1 -- 172.21.15.89:0/1555034826 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc007640 con 0x7f70d0004d90 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ebc02700 1 -- 172.21.15.89:0/1555034826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 msgr2=0x7f70d0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ebc02700 1 --2- 172.21.15.89:0/1555034826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70d0005200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00a760 tx=0x7f70d0005880 comp rx=0 tx=0).stop 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ea19f700 1 -- 172.21.15.89:0/1555034826 reap_dead start 2023-07-06T02:23:07.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ebc02700 1 -- 172.21.15.89:0/1555034826 shutdown_connections 2023-07-06T02:23:07.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ebc02700 1 -- 172.21.15.89:0/1555034826 >> 172.21.15.89:0/1555034826 conn(0x7f70e40cc490 msgr2=0x7f70e40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:07.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ebc02700 1 -- 172.21.15.89:0/1555034826 shutdown_connections 2023-07-06T02:23:07.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.279+0000 7f70ebc02700 1 -- 172.21.15.89:0/1555034826 wait complete. 2023-07-06T02:23:07.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 Processor -- start 2023-07-06T02:23:07.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 -- start start 2023-07-06T02:23:07.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70e40c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e4148150 0x7f70e40b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41485c0 0x7f70e40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41f62f0 con 0x7f70d0004d90 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41f5420 con 0x7f70e4148150 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70ebc02700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41f51f0 con 0x7f70e41485c0 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70e999e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70e40c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70e999e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70e40c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60578/0 (socket says 172.21.15.89:60578) 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.281+0000 7f70e999e700 1 -- 172.21.15.89:0/2985319241 learned_addr learned my addr 172.21.15.89:0/2985319241 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:07.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70ea19f700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41485c0 0x7f70e40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e919d700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e4148150 0x7f70e40b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e999e700 1 -- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41485c0 msgr2=0x7f70e40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e999e700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41485c0 0x7f70e40b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e999e700 1 -- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e4148150 msgr2=0x7f70e40b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e999e700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e4148150 0x7f70e40b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e999e700 1 -- 172.21.15.89:0/2985319241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e41f5670 con 0x7f70d0004d90 2023-07-06T02:23:07.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e919d700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e4148150 0x7f70e40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70ea19f700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41485c0 0x7f70e40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70e999e700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70e40c71b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70d400f820 tx=0x7f70d400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70da7fc700 1 -- 172.21.15.89:0/2985319241 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40052e0 con 0x7f70d0004d90 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70da7fc700 1 -- 172.21.15.89:0/2985319241 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4005440 con 0x7f70d0004d90 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e41f5950 con 0x7f70d0004d90 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70da7fc700 1 -- 172.21.15.89:0/2985319241 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40038c0 con 0x7f70d0004d90 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.282+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e41f5f10 con 0x7f70d0004d90 2023-07-06T02:23:07.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.283+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e41f60d0 con 0x7f70d0004d90 2023-07-06T02:23:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.283+0000 7f70da7fc700 1 -- 172.21.15.89:0/2985319241 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70d4003a20 con 0x7f70d0004d90 2023-07-06T02:23:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.284+0000 7f70da7fc700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70c806ace0 0x7f70c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.284+0000 7f70da7fc700 1 -- 172.21.15.89:0/2985319241 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f70d4089540 con 0x7f70d0004d90 2023-07-06T02:23:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.284+0000 7f70e919d700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70c806ace0 0x7f70c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.284+0000 7f70e919d700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70c806ace0 0x7f70c806d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70e41457b0 tx=0x7f70e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.288+0000 7f70da7fc700 1 -- 172.21.15.89:0/2985319241 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70d4054300 con 0x7f70d0004d90 2023-07-06T02:23:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.477+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70e40c4590 con 0x7f70d0004d90 2023-07-06T02:23:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.477+0000 7f70da7fc700 1 -- 172.21.15.89:0/2985319241 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70d4057920 con 0x7f70d0004d90 2023-07-06T02:23:07.479 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:07.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70c806ace0 msgr2=0x7f70c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70c806ace0 0x7f70c806d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70e41457b0 tx=0x7f70e0008040 comp rx=0 tx=0).stop 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 msgr2=0x7f70e40c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70e40c71b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70d400f820 tx=0x7f70d400fbe0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 shutdown_connections 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e41485c0 0x7f70e40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70c806ace0 0x7f70c806d190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d0004d90 0x7f70e40c71b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.479+0000 7f70ebc02700 1 --2- 172.21.15.89:0/2985319241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e4148150 0x7f70e40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.480+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 >> 172.21.15.89:0/2985319241 conn(0x7f70e40cc490 msgr2=0x7f70e40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.480+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 shutdown_connections 2023-07-06T02:23:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.480+0000 7f70ebc02700 1 -- 172.21.15.89:0/2985319241 wait complete. 2023-07-06T02:23:07.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.614+0000 7f9f3770a700 1 Processor -- start 2023-07-06T02:23:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.614+0000 7f9f3770a700 1 -- start start 2023-07-06T02:23:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f3770a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f3015c610 0x7f9f300cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f3770a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f300b5e50 0x7f9f30070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f3770a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f30070870 0x7f9f300ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f3770a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f30070cf0 con 0x7f9f300b5e50 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f3770a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f300cb150 con 0x7f9f30070870 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f3770a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300cb430 con 0x7f9f3015c610 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f300b5e50 0x7f9f30070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f300b5e50 0x7f9f30070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60584/0 (socket says 172.21.15.89:60584) 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 -- 172.21.15.89:0/2115424577 learned_addr learned my addr 172.21.15.89:0/2115424577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:07.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f354a6700 1 --2- 172.21.15.89:0/2115424577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f3015c610 0x7f9f300cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f35ca7700 1 --2- 172.21.15.89:0/2115424577 >> v2:172.21.15.172:3300/0 conn(0x7f9f30070870 0x7f9f300ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 -- 172.21.15.89:0/2115424577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f3015c610 msgr2=0x7f9f300cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 --2- 172.21.15.89:0/2115424577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f3015c610 0x7f9f300cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 -- 172.21.15.89:0/2115424577 >> v2:172.21.15.172:3300/0 conn(0x7f9f30070870 msgr2=0x7f9f300ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 --2- 172.21.15.89:0/2115424577 >> v2:172.21.15.172:3300/0 conn(0x7f9f30070870 0x7f9f300ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.615+0000 7f9f34ca5700 1 -- 172.21.15.89:0/2115424577 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300b6310 con 0x7f9f300b5e50 2023-07-06T02:23:07.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f34ca5700 1 --2- 172.21.15.89:0/2115424577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f300b5e50 0x7f9f30070330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f30072720 tx=0x7f9f2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1a64df5f42ec08 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f27fff700 1 -- 172.21.15.89:0/2115424577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00b870 con 0x7f9f300b5e50 2023-07-06T02:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f27fff700 1 -- 172.21.15.89:0/2115424577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c00b9d0 con 0x7f9f300b5e50 2023-07-06T02:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f27fff700 1 -- 172.21.15.89:0/2115424577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00bd60 con 0x7f9f300b5e50 2023-07-06T02:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 -- 172.21.15.89:0/2115424577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f300b5e50 msgr2=0x7f9f30070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 --2- 172.21.15.89:0/2115424577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f300b5e50 0x7f9f30070330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f30072720 tx=0x7f9f2c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 -- 172.21.15.89:0/2115424577 shutdown_connections 2023-07-06T02:23:07.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 --2- 172.21.15.89:0/2115424577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f3015c610 0x7f9f300cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 --2- 172.21.15.89:0/2115424577 >> v2:172.21.15.172:3300/0 conn(0x7f9f30070870 0x7f9f300ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 --2- 172.21.15.89:0/2115424577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f300b5e50 0x7f9f30070330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 -- 172.21.15.89:0/2115424577 >> 172.21.15.89:0/2115424577 conn(0x7f9f300d0b00 msgr2=0x7f9f300d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 -- 172.21.15.89:0/2115424577 shutdown_connections 2023-07-06T02:23:07.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.616+0000 7f9f3770a700 1 -- 172.21.15.89:0/2115424577 wait complete. 2023-07-06T02:23:07.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 Processor -- start 2023-07-06T02:23:07.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 -- start start 2023-07-06T02:23:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f30070870 0x7f9f3014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f300b5e50 0x7f9f3014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3015c610 0x7f9f3015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300c8ba0 con 0x7f9f3015c610 2023-07-06T02:23:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300c7f90 con 0x7f9f30070870 2023-07-06T02:23:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f3770a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300c8270 con 0x7f9f300b5e50 2023-07-06T02:23:07.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.618+0000 7f9f35ca7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3015c610 0x7f9f3015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3015c610 0x7f9f3015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60594/0 (socket says 172.21.15.89:60594) 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 -- 172.21.15.89:0/1132418931 learned_addr learned my addr 172.21.15.89:0/1132418931 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f354a6700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f30070870 0x7f9f3014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f34ca5700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f300b5e50 0x7f9f3014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 -- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f300b5e50 msgr2=0x7f9f3014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f300b5e50 0x7f9f3014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 -- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f30070870 msgr2=0x7f9f3014ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f30070870 0x7f9f3014ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 -- 172.21.15.89:0/1132418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c009550 con 0x7f9f3015c610 2023-07-06T02:23:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f35ca7700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3015c610 0x7f9f3015b2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f28007e40 tx=0x7f9f2800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f25ffb700 1 -- 172.21.15.89:0/1132418931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800ed30 con 0x7f9f3015c610 2023-07-06T02:23:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f25ffb700 1 -- 172.21.15.89:0/1132418931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f28014420 con 0x7f9f3015c610 2023-07-06T02:23:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f25ffb700 1 -- 172.21.15.89:0/1132418931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280147d0 con 0x7f9f3015c610 2023-07-06T02:23:07.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f300c8550 con 0x7f9f3015c610 2023-07-06T02:23:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.619+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f300c7bd0 con 0x7f9f3015c610 2023-07-06T02:23:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.620+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f300c7d60 con 0x7f9f3015c610 2023-07-06T02:23:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.621+0000 7f9f25ffb700 1 -- 172.21.15.89:0/1132418931 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f28014930 con 0x7f9f3015c610 2023-07-06T02:23:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.621+0000 7f9f25ffb700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9f1406ace0 0x7f9f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.621+0000 7f9f25ffb700 1 -- 172.21.15.89:0/1132418931 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9f28089e60 con 0x7f9f3015c610 2023-07-06T02:23:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.621+0000 7f9f354a6700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9f1406ace0 0x7f9f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.621+0000 7f9f354a6700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9f1406ace0 0x7f9f1406d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f3014f7f0 tx=0x7f9f20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.625+0000 7f9f25ffb700 1 -- 172.21.15.89:0/1132418931 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f28054cd0 con 0x7f9f3015c610 2023-07-06T02:23:07.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.813+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9f30070330 con 0x7f9f3015c610 2023-07-06T02:23:07.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.814+0000 7f9f25ffb700 1 -- 172.21.15.89:0/1132418931 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9f280582f0 con 0x7f9f3015c610 2023-07-06T02:23:07.816 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9f1406ace0 msgr2=0x7f9f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9f1406ace0 0x7f9f1406d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f3014f7f0 tx=0x7f9f20008040 comp rx=0 tx=0).stop 2023-07-06T02:23:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3015c610 msgr2=0x7f9f3015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3015c610 0x7f9f3015b2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f28007e40 tx=0x7f9f2800e3c0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 shutdown_connections 2023-07-06T02:23:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9f1406ace0 0x7f9f1406d190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3015c610 0x7f9f3015b2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f30070870 0x7f9f3014ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 --2- 172.21.15.89:0/1132418931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f300b5e50 0x7f9f3014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 >> 172.21.15.89:0/1132418931 conn(0x7f9f300d0b00 msgr2=0x7f9f30159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 shutdown_connections 2023-07-06T02:23:07.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.816+0000 7f9f3770a700 1 -- 172.21.15.89:0/1132418931 wait complete. 2023-07-06T02:23:07.830 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T02:23:07.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.949+0000 7f78fca4b700 1 Processor -- start 2023-07-06T02:23:07.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.949+0000 7f78fca4b700 1 -- start start 2023-07-06T02:23:07.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.950+0000 7f78fca4b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f78f815c690 0x7f78f80cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.950+0000 7f78fca4b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 0x7f78f80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.950+0000 7f78fca4b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f80cab50 con 0x7f78f80b19d0 2023-07-06T02:23:07.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.950+0000 7f78fca4b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f815de50 con 0x7f78f815c690 2023-07-06T02:23:07.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.950+0000 7f78fca4b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f815db70 con 0x7f78f80cbda0 2023-07-06T02:23:07.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.950+0000 7f78f6d9d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f78f80b19d0 0x7f78f806bfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57268/0 (socket says 172.21.15.89:57268) 2023-07-06T02:23:07.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.950+0000 7f78f6d9d700 1 -- 172.21.15.89:0/2778810798 learned_addr learned my addr 172.21.15.89:0/2778810798 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:07.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f659c700 1 --2- 172.21.15.89:0/2778810798 >> v2:172.21.15.172:3300/0 conn(0x7f78f815c690 0x7f78f80cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 0x7f78f80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629011179 0 0) 0x7f78f80cab50 con 0x7f78f80b19d0 2023-07-06T02:23:07.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e00035b0 con 0x7f78f80b19d0 2023-07-06T02:23:07.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217006258 0 0) 0x7f78e00035b0 con 0x7f78f80b19d0 2023-07-06T02:23:07.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 msgr2=0x7f78f80b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 0x7f78f80b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 >> v2:172.21.15.172:3300/0 conn(0x7f78f815c690 msgr2=0x7f78f80cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 --2- 172.21.15.89:0/2778810798 >> v2:172.21.15.172:3300/0 conn(0x7f78f815c690 0x7f78f80cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80b1e30 con 0x7f78f80b19d0 2023-07-06T02:23:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f78f00019f0 con 0x7f78f80b19d0 2023-07-06T02:23:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 >> v1:172.21.15.89:6789/0 conn(0x7f78f80b19d0 legacy=0x7f78f806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e0003a00 0x7f78e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78e0004430 0x7f78e00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 0x7f78e0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f815de50 con 0x7f78e0004430 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f815db70 con 0x7f78e0003a00 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e000ba10 con 0x7f78f80cbda0 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78e0004430 0x7f78e00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f659c700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e0003a00 0x7f78e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f6d9d700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 0x7f78e0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f5d9b700 1 -- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 msgr2=0x7f78e0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 0x7f78e0004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f5d9b700 1 -- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e0003a00 msgr2=0x7f78e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e0003a00 0x7f78e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f5d9b700 1 -- 172.21.15.89:0/2778810798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e00035b0 con 0x7f78e0004430 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.951+0000 7f78f6d9d700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78f80cbda0 0x7f78e0004fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78e0004430 0x7f78e00048f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009b60 tx=0x7f78e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b9398527fd7987 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800e6c0 con 0x7f78e0004430 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e800e820 con 0x7f78e0004430 2023-07-06T02:23:07.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78f559a700 1 -- 172.21.15.89:0/2778810798 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800eb90 con 0x7f78e0004430 2023-07-06T02:23:07.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78fca4b700 1 -- 172.21.15.89:0/2778810798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78e0004430 msgr2=0x7f78e00048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78fca4b700 1 --2- 172.21.15.89:0/2778810798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78e0004430 0x7f78e00048f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009b60 tx=0x7f78e0005880 comp rx=0 tx=0).stop 2023-07-06T02:23:07.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78f6d9d700 1 -- 172.21.15.89:0/2778810798 reap_dead start 2023-07-06T02:23:07.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78fca4b700 1 -- 172.21.15.89:0/2778810798 shutdown_connections 2023-07-06T02:23:07.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78fca4b700 1 -- 172.21.15.89:0/2778810798 >> 172.21.15.89:0/2778810798 conn(0x7f78f80cc490 msgr2=0x7f78f80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:07.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78fca4b700 1 -- 172.21.15.89:0/2778810798 shutdown_connections 2023-07-06T02:23:07.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.952+0000 7f78fca4b700 1 -- 172.21.15.89:0/2778810798 wait complete. 2023-07-06T02:23:07.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 Processor -- start 2023-07-06T02:23:07.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 -- start start 2023-07-06T02:23:07.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78e0004430 0x7f78f80c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f80a74e0 0x7f78f80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78f81595b0 0x7f78f8159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80a8b20 con 0x7f78f80a74e0 2023-07-06T02:23:07.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80a7a80 con 0x7f78f81595b0 2023-07-06T02:23:07.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.954+0000 7f78fca4b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80a7d60 con 0x7f78e0004430 2023-07-06T02:23:07.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f80a74e0 0x7f78f80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f80a74e0 0x7f78f80cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60612/0 (socket says 172.21.15.89:60612) 2023-07-06T02:23:07.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 -- 172.21.15.89:0/2308569471 learned_addr learned my addr 172.21.15.89:0/2308569471 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:07.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f659c700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78e0004430 0x7f78f80c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f6d9d700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78f81595b0 0x7f78f8159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 -- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78e0004430 msgr2=0x7f78f80c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78e0004430 0x7f78f80c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 -- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78f81595b0 msgr2=0x7f78f8159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78f81595b0 0x7f78f8159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 -- 172.21.15.89:0/2308569471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80a8040 con 0x7f78f80a74e0 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78f5d9b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f80a74e0 0x7f78f80cfc10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78e00056d0 tx=0x7f78e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78e6ffd700 1 -- 172.21.15.89:0/2308569471 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8021070 con 0x7f78f80a74e0 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78e6ffd700 1 -- 172.21.15.89:0/2308569471 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e80045d0 con 0x7f78f80a74e0 2023-07-06T02:23:07.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78e6ffd700 1 -- 172.21.15.89:0/2308569471 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800e6c0 con 0x7f78f80a74e0 2023-07-06T02:23:07.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f80a8320 con 0x7f78f80a74e0 2023-07-06T02:23:07.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.955+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f80c5910 con 0x7f78f80a74e0 2023-07-06T02:23:07.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.956+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f81f1270 con 0x7f78f80a74e0 2023-07-06T02:23:07.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.957+0000 7f78e6ffd700 1 -- 172.21.15.89:0/2308569471 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78e8025070 con 0x7f78f80a74e0 2023-07-06T02:23:07.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.957+0000 7f78e6ffd700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f78d806ace0 0x7f78d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:07.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.957+0000 7f78e6ffd700 1 -- 172.21.15.89:0/2308569471 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f78e808a7c0 con 0x7f78f80a74e0 2023-07-06T02:23:07.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.957+0000 7f78f659c700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f78d806ace0 0x7f78d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:07.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.957+0000 7f78f659c700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f78d806ace0 0x7f78d806d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f78f80b0870 tx=0x7f78ec009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:07.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:07.961+0000 7f78e6ffd700 1 -- 172.21.15.89:0/2308569471 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f78e80557f0 con 0x7f78f80a74e0 2023-07-06T02:23:08.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.165+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78f8074270 con 0x7f78f80a74e0 2023-07-06T02:23:08.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.165+0000 7f78e6ffd700 1 -- 172.21.15.89:0/2308569471 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f78e8058e10 con 0x7f78f80a74e0 2023-07-06T02:23:08.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f78d806ace0 msgr2=0x7f78d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f78d806ace0 0x7f78d806d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f78f80b0870 tx=0x7f78ec009210 comp rx=0 tx=0).stop 2023-07-06T02:23:08.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f80a74e0 msgr2=0x7f78f80cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f80a74e0 0x7f78f80cfc10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78e00056d0 tx=0x7f78e80047c0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 shutdown_connections 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78f81595b0 0x7f78f8159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78e0004430 0x7f78f80c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f78d806ace0 0x7f78d806d190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 --2- 172.21.15.89:0/2308569471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78f80a74e0 0x7f78f80cfc10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 >> 172.21.15.89:0/2308569471 conn(0x7f78f80cc490 msgr2=0x7f78f815d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 shutdown_connections 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.167+0000 7f78fca4b700 1 -- 172.21.15.89:0/2308569471 wait complete. 2023-07-06T02:23:08.171 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-07-06T02:23:08.181 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi089.front.sepia.ceph.com... 2023-07-06T02:23:08.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:08.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-06T02:23:08.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[139278]: fuse finished with error 0 and tester_r 0 2023-07-06T02:23:08.218 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T02:23:08.235 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:23:08.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:08.236 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-06T02:23:08.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:08.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.455+0000 7f1fb5259700 1 Processor -- start 2023-07-06T02:23:08.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.455+0000 7f1fb5259700 1 -- start start 2023-07-06T02:23:08.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1fb5259700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1fb00cecf0 0x7f1fb00ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1fb5259700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 0x7f1fb006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1fb5259700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb006ebe0 con 0x7f1fb00c9ee0 2023-07-06T02:23:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1fb5259700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb006abe0 con 0x7f1fb00cecf0 2023-07-06T02:23:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1fb5259700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb006b160 con 0x7f1fb006e250 2023-07-06T02:23:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1faf59e700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1fb00c9ee0 0x7f1fb00ca340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57288/0 (socket says 172.21.15.89:57288) 2023-07-06T02:23:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1faf59e700 1 -- 172.21.15.89:0/4292339302 learned_addr learned my addr 172.21.15.89:0/4292339302 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708853956 0 0) 0x7f1fa8001490 con 0x7f1fb00c9ee0 2023-07-06T02:23:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.456+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f980035b0 con 0x7f1fb00c9ee0 2023-07-06T02:23:08.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 0x7f1fb006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1faed9d700 1 --2- 172.21.15.89:0/4292339302 >> v2:172.21.15.172:3300/0 conn(0x7f1fb00cecf0 0x7f1fb00ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087683602 0 0) 0x7f1f980035b0 con 0x7f1fb00c9ee0 2023-07-06T02:23:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 msgr2=0x7f1fb006e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 0x7f1fb006e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 >> v2:172.21.15.172:3300/0 conn(0x7f1fb00cecf0 msgr2=0x7f1fb00ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 --2- 172.21.15.89:0/4292339302 >> v2:172.21.15.172:3300/0 conn(0x7f1fb00cecf0 0x7f1fb00ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00cac10 con 0x7f1fb00c9ee0 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f1fa8001b80 con 0x7f1fb00c9ee0 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 >> v1:172.21.15.89:6789/0 conn(0x7f1fb00c9ee0 legacy=0x7f1fb00ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f98003a00 0x7f1f98003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f98004850 0x7f1f98004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 0x7f1f98005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb006abe0 con 0x7f1f98004850 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb006b160 con 0x7f1f98003a00 2023-07-06T02:23:08.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9800bac0 con 0x7f1fb006e250 2023-07-06T02:23:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f98004850 0x7f1f98004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1faed9d700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f98003a00 0x7f1f98003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1faf59e700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 0x7f1f98005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 -- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 msgr2=0x7f1f98005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:08.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb006e250 0x7f1f98005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 -- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f98003a00 msgr2=0x7f1f98003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f98003a00 0x7f1f98003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 -- 172.21.15.89:0/4292339302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f980035b0 con 0x7f1f98004850 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1faed9d700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f98003a00 0x7f1f98003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.457+0000 7f1fae59c700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f98004850 0x7f1f98004cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009890 tx=0x7f1f98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2869fe5c6bcd88fe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000c070 con 0x7f1f98004850 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0013610 con 0x7f1f98004850 2023-07-06T02:23:08.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fadd9b700 1 -- 172.21.15.89:0/4292339302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00139a0 con 0x7f1f98004850 2023-07-06T02:23:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fb5259700 1 -- 172.21.15.89:0/4292339302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f98004850 msgr2=0x7f1f98004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fb5259700 1 --2- 172.21.15.89:0/4292339302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f98004850 0x7f1f98004cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009890 tx=0x7f1f98005880 comp rx=0 tx=0).stop 2023-07-06T02:23:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1faf59e700 1 -- 172.21.15.89:0/4292339302 reap_dead start 2023-07-06T02:23:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fb5259700 1 -- 172.21.15.89:0/4292339302 shutdown_connections 2023-07-06T02:23:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fb5259700 1 -- 172.21.15.89:0/4292339302 >> 172.21.15.89:0/4292339302 conn(0x7f1fb00cf1c0 msgr2=0x7f1fb00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fb5259700 1 -- 172.21.15.89:0/4292339302 shutdown_connections 2023-07-06T02:23:08.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.458+0000 7f1fb5259700 1 -- 172.21.15.89:0/4292339302 wait complete. 2023-07-06T02:23:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.460+0000 7f1fb5259700 1 Processor -- start 2023-07-06T02:23:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.460+0000 7f1fb5259700 1 -- start start 2023-07-06T02:23:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fb5259700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb0152240 0x7f1fb00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fb5259700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fb00b0000 0x7f1fb00c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fb5259700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fb00c9250 0x7f1fb01f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fb5259700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01f39b0 con 0x7f1fb00b0000 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fb5259700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb0152b90 con 0x7f1fb00c9250 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fb5259700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb00b4820 con 0x7f1fb0152240 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fb00b0000 0x7f1fb00c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fb00b0000 0x7f1fb00c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60632/0 (socket says 172.21.15.89:60632) 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1fae59c700 1 -- 172.21.15.89:0/3650895761 learned_addr learned my addr 172.21.15.89:0/3650895761 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1faed9d700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb0152240 0x7f1fb00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.461+0000 7f1faf59e700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fb00c9250 0x7f1fb01f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fae59c700 1 -- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb0152240 msgr2=0x7f1fb00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fae59c700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb0152240 0x7f1fb00afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fae59c700 1 -- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fb00c9250 msgr2=0x7f1fb01f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fae59c700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fb00c9250 0x7f1fb01f3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fae59c700 1 -- 172.21.15.89:0/3650895761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00b4b00 con 0x7f1fb00b0000 2023-07-06T02:23:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fae59c700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fb00b0000 0x7f1fb00c8d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0008010 tx=0x7f1fa00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:08.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1f977fe700 1 -- 172.21.15.89:0/3650895761 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0013cd0 con 0x7f1fb00b0000 2023-07-06T02:23:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1f977fe700 1 -- 172.21.15.89:0/3650895761 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0013e50 con 0x7f1fb00b0000 2023-07-06T02:23:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb00b4de0 con 0x7f1fb00b0000 2023-07-06T02:23:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1f977fe700 1 -- 172.21.15.89:0/3650895761 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00038e0 con 0x7f1fb00b0000 2023-07-06T02:23:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.462+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb0152610 con 0x7f1fb00b0000 2023-07-06T02:23:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.463+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb0074270 con 0x7f1fb00b0000 2023-07-06T02:23:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.463+0000 7f1f977fe700 1 -- 172.21.15.89:0/3650895761 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fa0003a40 con 0x7f1fb00b0000 2023-07-06T02:23:08.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.463+0000 7f1f977fe700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f9c06ace0 0x7f1f9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.464+0000 7f1f977fe700 1 -- 172.21.15.89:0/3650895761 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1fa008aa10 con 0x7f1fb00b0000 2023-07-06T02:23:08.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.464+0000 7f1faed9d700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f9c06ace0 0x7f1f9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.464+0000 7f1faed9d700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f9c06ace0 0x7f1f9c06d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009870 tx=0x7f1fa4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:08.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.468+0000 7f1f977fe700 1 -- 172.21.15.89:0/3650895761 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fa0055860 con 0x7f1fb00b0000 2023-07-06T02:23:08.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.655+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fb006f5f0 con 0x7f1fb00b0000 2023-07-06T02:23:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.656+0000 7f1f977fe700 1 -- 172.21.15.89:0/3650895761 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f1fa0058e80 con 0x7f1fb00b0000 2023-07-06T02:23:08.659 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:08.659 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f9c06ace0 msgr2=0x7f1f9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f9c06ace0 0x7f1f9c06d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009870 tx=0x7f1fa4008040 comp rx=0 tx=0).stop 2023-07-06T02:23:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fb00b0000 msgr2=0x7f1fb00c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fb00b0000 0x7f1fb00c8d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0008010 tx=0x7f1fa00047c0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 shutdown_connections 2023-07-06T02:23:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fb00c9250 0x7f1fb01f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fb0152240 0x7f1fb00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f9c06ace0 0x7f1f9c06d190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 --2- 172.21.15.89:0/3650895761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fb00b0000 0x7f1fb00c8d10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 >> 172.21.15.89:0/3650895761 conn(0x7f1fb00cf1c0 msgr2=0x7f1fb01555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 shutdown_connections 2023-07-06T02:23:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.659+0000 7f1fb5259700 1 -- 172.21.15.89:0/3650895761 wait complete. 2023-07-06T02:23:08.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:08.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.793+0000 7f7101ef3700 1 Processor -- start 2023-07-06T02:23:08.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.793+0000 7f7101ef3700 1 -- start start 2023-07-06T02:23:08.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f7101ef3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70fc0b1490 0x7f70fc0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f7101ef3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70fc06bfa0 0x7f70fc06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f7101ef3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0cab50 con 0x7f70fc15c690 2023-07-06T02:23:08.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f7101ef3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc15de50 con 0x7f70fc0b1490 2023-07-06T02:23:08.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f7101ef3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0b1e50 con 0x7f70fc06bfa0 2023-07-06T02:23:08.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70faffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70fc0b1490 0x7f70fc0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70fb7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f70fc15c690 0x7f70fc0cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57308/0 (socket says 172.21.15.89:57308) 2023-07-06T02:23:08.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70fb7fe700 1 -- 172.21.15.89:0/4003023548 learned_addr learned my addr 172.21.15.89:0/4003023548 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:08.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70faffd700 1 -- 172.21.15.89:0/4003023548 >> v2:172.21.15.172:3300/0 conn(0x7f70fc06bfa0 msgr2=0x7f70fc06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:08.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70faffd700 1 --2- 172.21.15.89:0/4003023548 >> v2:172.21.15.172:3300/0 conn(0x7f70fc06bfa0 0x7f70fc06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70faffd700 1 -- 172.21.15.89:0/4003023548 >> v1:172.21.15.89:6789/0 conn(0x7f70fc15c690 legacy=0x7f70fc0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70faffd700 1 -- 172.21.15.89:0/4003023548 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc06c8e0 con 0x7f70fc0b1490 2023-07-06T02:23:08.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70fa7fc700 1 -- 172.21.15.89:0/4003023548 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213109876 0 0) 0x7f70fc0cab50 con 0x7f70fc15c690 2023-07-06T02:23:08.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.794+0000 7f70faffd700 1 --2- 172.21.15.89:0/4003023548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc0b1490 0x7f70fc0b1910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0af420 tx=0x7f70f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27c1094b45c2068 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:08.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f70fa7fc700 1 -- 172.21.15.89:0/4003023548 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0011830 con 0x7f70fc0b1490 2023-07-06T02:23:08.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f70fa7fc700 1 -- 172.21.15.89:0/4003023548 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0011990 con 0x7f70fc0b1490 2023-07-06T02:23:08.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f70fa7fc700 1 -- 172.21.15.89:0/4003023548 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0011d20 con 0x7f70fc0b1490 2023-07-06T02:23:08.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 -- 172.21.15.89:0/4003023548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc0b1490 msgr2=0x7f70fc0b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 --2- 172.21.15.89:0/4003023548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc0b1490 0x7f70fc0b1910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0af420 tx=0x7f70f0009b20 comp rx=0 tx=0).stop 2023-07-06T02:23:08.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 -- 172.21.15.89:0/4003023548 shutdown_connections 2023-07-06T02:23:08.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 --2- 172.21.15.89:0/4003023548 >> v2:172.21.15.172:3300/0 conn(0x7f70fc06bfa0 0x7f70fc06c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 --2- 172.21.15.89:0/4003023548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc0b1490 0x7f70fc0b1910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 -- 172.21.15.89:0/4003023548 >> 172.21.15.89:0/4003023548 conn(0x7f70fc0cc490 msgr2=0x7f70fc0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:08.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 -- 172.21.15.89:0/4003023548 shutdown_connections 2023-07-06T02:23:08.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.795+0000 7f7101ef3700 1 -- 172.21.15.89:0/4003023548 wait complete. 2023-07-06T02:23:08.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.796+0000 7f7101ef3700 1 Processor -- start 2023-07-06T02:23:08.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f7101ef3700 1 -- start start 2023-07-06T02:23:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f7101ef3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc06bfa0 0x7f70fc1f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f7101ef3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70fc0b1490 0x7f70fc14d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f7101ef3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70fc15c690 0x7f70fc1f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f7101ef3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1f32b0 con 0x7f70fc06bfa0 2023-07-06T02:23:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f7101ef3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0c71e0 con 0x7f70fc15c690 2023-07-06T02:23:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f7101ef3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1f5e80 con 0x7f70fc0b1490 2023-07-06T02:23:08.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f70fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc06bfa0 0x7f70fc1f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f70fb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc06bfa0 0x7f70fc1f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60644/0 (socket says 172.21.15.89:60644) 2023-07-06T02:23:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.797+0000 7f70fb7fe700 1 -- 172.21.15.89:0/3165453379 learned_addr learned my addr 172.21.15.89:0/3165453379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fbfff700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70fc15c690 0x7f70fc1f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70faffd700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70fc0b1490 0x7f70fc14d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fb7fe700 1 -- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70fc0b1490 msgr2=0x7f70fc14d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fb7fe700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70fc0b1490 0x7f70fc14d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fb7fe700 1 -- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70fc15c690 msgr2=0x7f70fc1f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fb7fe700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70fc15c690 0x7f70fc1f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fb7fe700 1 -- 172.21.15.89:0/3165453379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00094a0 con 0x7f70fc06bfa0 2023-07-06T02:23:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fbfff700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70fc15c690 0x7f70fc1f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70faffd700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70fc0b1490 0x7f70fc14d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70fb7fe700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc06bfa0 0x7f70fc1f58a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c750 tx=0x7f70ec00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70e3fff700 1 -- 172.21.15.89:0/3165453379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec004c80 con 0x7f70fc06bfa0 2023-07-06T02:23:08.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70e3fff700 1 -- 172.21.15.89:0/3165453379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec004e00 con 0x7f70fc06bfa0 2023-07-06T02:23:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc1f6160 con 0x7f70fc06bfa0 2023-07-06T02:23:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f70e3fff700 1 -- 172.21.15.89:0/3165453379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec007310 con 0x7f70fc06bfa0 2023-07-06T02:23:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.798+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc1f6720 con 0x7f70fc06bfa0 2023-07-06T02:23:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.799+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc074270 con 0x7f70fc06bfa0 2023-07-06T02:23:08.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.799+0000 7f70e3fff700 1 -- 172.21.15.89:0/3165453379 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70ec007470 con 0x7f70fc06bfa0 2023-07-06T02:23:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.800+0000 7f70e3fff700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70e806ace0 0x7f70e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.800+0000 7f70e3fff700 1 -- 172.21.15.89:0/3165453379 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f70ec089480 con 0x7f70fc06bfa0 2023-07-06T02:23:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.800+0000 7f70faffd700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70e806ace0 0x7f70e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:08.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.800+0000 7f70faffd700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70e806ace0 0x7f70e806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0c9ae0 tx=0x7f70f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:08.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.804+0000 7f70e3fff700 1 -- 172.21.15.89:0/3165453379 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70ec0542d0 con 0x7f70fc06bfa0 2023-07-06T02:23:08.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.992+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70fc0cf0c0 con 0x7f70fc06bfa0 2023-07-06T02:23:08.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.993+0000 7f70e3fff700 1 -- 172.21.15.89:0/3165453379 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f70ec0578f0 con 0x7f70fc06bfa0 2023-07-06T02:23:08.997 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:08.997 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:08.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70e806ace0 msgr2=0x7f70e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70e806ace0 0x7f70e806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0c9ae0 tx=0x7f70f0008030 comp rx=0 tx=0).stop 2023-07-06T02:23:08.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc06bfa0 msgr2=0x7f70fc1f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:08.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc06bfa0 0x7f70fc1f58a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c750 tx=0x7f70ec00cb10 comp rx=0 tx=0).stop 2023-07-06T02:23:08.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 shutdown_connections 2023-07-06T02:23:08.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70fc15c690 0x7f70fc1f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f70e806ace0 0x7f70e806d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:08.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc06bfa0 0x7f70fc1f58a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 --2- 172.21.15.89:0/3165453379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70fc0b1490 0x7f70fc14d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.995+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 >> 172.21.15.89:0/3165453379 conn(0x7f70fc0cc490 msgr2=0x7f70fc0cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:09.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.996+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 shutdown_connections 2023-07-06T02:23:09.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:08.996+0000 7f7101ef3700 1 -- 172.21.15.89:0/3165453379 wait complete. 2023-07-06T02:23:09.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T02:23:09.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.130+0000 7f9c38a34700 1 Processor -- start 2023-07-06T02:23:09.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.130+0000 7f9c38a34700 1 -- start start 2023-07-06T02:23:09.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c38a34700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c34070f20 0x7f9c340c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:09.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c38a34700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c340712f0 0x7f9c3406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:09.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c38a34700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340c9fa0 con 0x7f9c3406e790 2023-07-06T02:23:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c38a34700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3406f290 con 0x7f9c34070f20 2023-07-06T02:23:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c38a34700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3406ecd0 con 0x7f9c340712f0 2023-07-06T02:23:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c3259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c34070f20 0x7f9c340c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c3259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9c34070f20 0x7f9c340c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60652/0 (socket says 172.21.15.89:60652) 2023-07-06T02:23:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c3259c700 1 -- 172.21.15.89:0/1905699158 learned_addr learned my addr 172.21.15.89:0/1905699158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:09.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c31d9b700 1 --2- 172.21.15.89:0/1905699158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c340712f0 0x7f9c3406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c3259c700 1 -- 172.21.15.89:0/1905699158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c340712f0 msgr2=0x7f9c3406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c3259c700 1 --2- 172.21.15.89:0/1905699158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c340712f0 0x7f9c3406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c3259c700 1 -- 172.21.15.89:0/1905699158 >> v1:172.21.15.172:6789/0 conn(0x7f9c3406e790 legacy=0x7f9c3406b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.131+0000 7f9c3259c700 1 -- 172.21.15.89:0/1905699158 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3406efb0 con 0x7f9c34070f20 2023-07-06T02:23:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c3259c700 1 --2- 172.21.15.89:0/1905699158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c34070f20 0x7f9c340c9a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800bb90 tx=0x7f9c2800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b94abeaed0221 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c3159a700 1 -- 172.21.15.89:0/1905699158 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800dc20 con 0x7f9c34070f20 2023-07-06T02:23:09.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c3159a700 1 -- 172.21.15.89:0/1905699158 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2800dd80 con 0x7f9c34070f20 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c3159a700 1 -- 172.21.15.89:0/1905699158 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28017630 con 0x7f9c34070f20 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 -- 172.21.15.89:0/1905699158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c34070f20 msgr2=0x7f9c340c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 --2- 172.21.15.89:0/1905699158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c34070f20 0x7f9c340c9a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800bb90 tx=0x7f9c2800bea0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 -- 172.21.15.89:0/1905699158 shutdown_connections 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 --2- 172.21.15.89:0/1905699158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c34070f20 0x7f9c340c9a60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 --2- 172.21.15.89:0/1905699158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c340712f0 0x7f9c3406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 -- 172.21.15.89:0/1905699158 >> 172.21.15.89:0/1905699158 conn(0x7f9c340cf1c0 msgr2=0x7f9c340cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:09.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 -- 172.21.15.89:0/1905699158 shutdown_connections 2023-07-06T02:23:09.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.132+0000 7f9c38a34700 1 -- 172.21.15.89:0/1905699158 wait complete. 2023-07-06T02:23:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 Processor -- start 2023-07-06T02:23:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 -- start start 2023-07-06T02:23:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c3406e790 0x7f9c340c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:09.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c34070f20 0x7f9c341eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:09.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c340712f0 0x7f9c340b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:09.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3414ff30 con 0x7f9c340712f0 2023-07-06T02:23:09.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340c2cf0 con 0x7f9c34070f20 2023-07-06T02:23:09.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.134+0000 7f9c38a34700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340c4b60 con 0x7f9c3406e790 2023-07-06T02:23:09.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c340712f0 0x7f9c340b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:09.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c3259c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c3406e790 0x7f9c340c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:09.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c31d9b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c34070f20 0x7f9c341eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c340712f0 0x7f9c340b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60660/0 (socket says 172.21.15.89:60660) 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 -- 172.21.15.89:0/617649937 learned_addr learned my addr 172.21.15.89:0/617649937 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 -- 172.21.15.89:0/617649937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c3406e790 msgr2=0x7f9c340c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c3406e790 0x7f9c340c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 -- 172.21.15.89:0/617649937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c34070f20 msgr2=0x7f9c341eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c34070f20 0x7f9c341eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 -- 172.21.15.89:0/617649937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2800b7f0 con 0x7f9c340712f0 2023-07-06T02:23:09.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c32d9d700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c340712f0 0x7f9c340b0110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00cb50 tx=0x7f9c2c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c22ffd700 1 -- 172.21.15.89:0/617649937 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c003c00 con 0x7f9c340712f0 2023-07-06T02:23:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c22ffd700 1 -- 172.21.15.89:0/617649937 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c003d60 con 0x7f9c340712f0 2023-07-06T02:23:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c341ef170 con 0x7f9c340712f0 2023-07-06T02:23:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c22ffd700 1 -- 172.21.15.89:0/617649937 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00aa20 con 0x7f9c340712f0 2023-07-06T02:23:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.135+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340c5650 con 0x7f9c340712f0 2023-07-06T02:23:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.136+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c340c5900 con 0x7f9c340712f0 2023-07-06T02:23:09.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.137+0000 7f9c22ffd700 1 -- 172.21.15.89:0/617649937 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c2c00ab80 con 0x7f9c340712f0 2023-07-06T02:23:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.137+0000 7f9c22ffd700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9c1406ace0 0x7f9c1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.137+0000 7f9c22ffd700 1 -- 172.21.15.89:0/617649937 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9c2c0896f0 con 0x7f9c340712f0 2023-07-06T02:23:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.137+0000 7f9c3259c700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9c1406ace0 0x7f9c1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.138+0000 7f9c3259c700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9c1406ace0 0x7f9c1406d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c28019040 tx=0x7f9c28014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:09.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.142+0000 7f9c22ffd700 1 -- 172.21.15.89:0/617649937 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9c2c054540 con 0x7f9c340712f0 2023-07-06T02:23:09.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:09.372+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9c34074270 con 0x7f9c340712f0 2023-07-06T02:23:10.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.178+0000 7f9c22ffd700 1 -- 172.21.15.89:0/617649937 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9c2c057b60 con 0x7f9c340712f0 2023-07-06T02:23:10.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9c1406ace0 msgr2=0x7f9c1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:10.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9c1406ace0 0x7f9c1406d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c28019040 tx=0x7f9c28014000 comp rx=0 tx=0).stop 2023-07-06T02:23:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c340712f0 msgr2=0x7f9c340b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c340712f0 0x7f9c340b0110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00cb50 tx=0x7f9c2c00ce60 comp rx=0 tx=0).stop 2023-07-06T02:23:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 shutdown_connections 2023-07-06T02:23:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9c1406ace0 0x7f9c1406d190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c340712f0 0x7f9c340b0110 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c3406e790 0x7f9c340c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 --2- 172.21.15.89:0/617649937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c34070f20 0x7f9c341eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 >> 172.21.15.89:0/617649937 conn(0x7f9c340cf1c0 msgr2=0x7f9c340b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 shutdown_connections 2023-07-06T02:23:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.180+0000 7f9c38a34700 1 -- 172.21.15.89:0/617649937 wait complete. 2023-07-06T02:23:10.198 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-07-06T02:23:10.198 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T02:23:10.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1ba967700 1 Processor -- start 2023-07-06T02:23:10.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1ba967700 1 -- start start 2023-07-06T02:23:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1ba967700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1b40705a0 0x7ff1b40ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1ba967700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1b40cd180 0x7ff1b415a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1ba967700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b406e110 con 0x7ff1b406dc10 2023-07-06T02:23:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1ba967700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b406e3f0 con 0x7ff1b40705a0 2023-07-06T02:23:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1ba967700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b406e6d0 con 0x7ff1b40cd180 2023-07-06T02:23:10.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.324+0000 7ff1b3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1b40705a0 0x7ff1b40ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b8f04700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff1b406dc10 0x7ff1b4157950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57334/0 (socket says 172.21.15.89:57334) 2023-07-06T02:23:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b8f04700 1 -- 172.21.15.89:0/3055399742 learned_addr learned my addr 172.21.15.89:0/3055399742 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b37fe700 1 --2- 172.21.15.89:0/3055399742 >> v2:172.21.15.172:3300/0 conn(0x7ff1b40cd180 0x7ff1b415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b3fff700 1 -- 172.21.15.89:0/3055399742 >> v2:172.21.15.172:3300/0 conn(0x7ff1b40cd180 msgr2=0x7ff1b415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b3fff700 1 --2- 172.21.15.89:0/3055399742 >> v2:172.21.15.172:3300/0 conn(0x7ff1b40cd180 0x7ff1b415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b3fff700 1 -- 172.21.15.89:0/3055399742 >> v1:172.21.15.89:6789/0 conn(0x7ff1b406dc10 legacy=0x7ff1b4157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:10.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b3fff700 1 -- 172.21.15.89:0/3055399742 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b406e9b0 con 0x7ff1b40705a0 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b2ffd700 1 -- 172.21.15.89:0/3055399742 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581641453 0 0) 0x7ff1b406e110 con 0x7ff1b406dc10 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b3fff700 1 --2- 172.21.15.89:0/3055399742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40705a0 0x7ff1b40ccc40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40cc4d0 tx=0x7ff1a4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c62e3390edde2f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b2ffd700 1 -- 172.21.15.89:0/3055399742 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4011830 con 0x7ff1b40705a0 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b2ffd700 1 -- 172.21.15.89:0/3055399742 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4011990 con 0x7ff1b40705a0 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.325+0000 7ff1b2ffd700 1 -- 172.21.15.89:0/3055399742 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4011d20 con 0x7ff1b40705a0 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 -- 172.21.15.89:0/3055399742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40705a0 msgr2=0x7ff1b40ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 --2- 172.21.15.89:0/3055399742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40705a0 0x7ff1b40ccc40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40cc4d0 tx=0x7ff1a4009b20 comp rx=0 tx=0).stop 2023-07-06T02:23:10.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 -- 172.21.15.89:0/3055399742 shutdown_connections 2023-07-06T02:23:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 --2- 172.21.15.89:0/3055399742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40705a0 0x7ff1b40ccc40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 --2- 172.21.15.89:0/3055399742 >> v2:172.21.15.172:3300/0 conn(0x7ff1b40cd180 0x7ff1b415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 -- 172.21.15.89:0/3055399742 >> 172.21.15.89:0/3055399742 conn(0x7ff1b40d0b00 msgr2=0x7ff1b40d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 -- 172.21.15.89:0/3055399742 shutdown_connections 2023-07-06T02:23:10.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.326+0000 7ff1ba967700 1 -- 172.21.15.89:0/3055399742 wait complete. 2023-07-06T02:23:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.327+0000 7ff1ba967700 1 Processor -- start 2023-07-06T02:23:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.327+0000 7ff1ba967700 1 -- start start 2023-07-06T02:23:10.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1ba967700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1b406dc10 0x7ff1b40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1ba967700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1b40705a0 0x7ff1b4158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1ba967700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40cd180 0x7ff1b414bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1ba967700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40b4db0 con 0x7ff1b40cd180 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1ba967700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40b4230 con 0x7ff1b406dc10 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1ba967700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40b4510 con 0x7ff1b40705a0 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40cd180 0x7ff1b414bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40cd180 0x7ff1b414bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60674/0 (socket says 172.21.15.89:60674) 2023-07-06T02:23:10.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 -- 172.21.15.89:0/2623613920 learned_addr learned my addr 172.21.15.89:0/2623613920 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b3fff700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1b406dc10 0x7ff1b40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b37fe700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1b40705a0 0x7ff1b4158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 -- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1b40705a0 msgr2=0x7ff1b4158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1b40705a0 0x7ff1b4158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 -- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1b406dc10 msgr2=0x7ff1b40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1b406dc10 0x7ff1b40b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:10.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 -- 172.21.15.89:0/2623613920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a40094a0 con 0x7ff1b40cd180 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b8f04700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40cd180 0x7ff1b414bb40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00cba0 tx=0x7ff1ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b0ff9700 1 -- 172.21.15.89:0/2623613920 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac00ab80 con 0x7ff1b40cd180 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b0ff9700 1 -- 172.21.15.89:0/2623613920 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac00ad00 con 0x7ff1b40cd180 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b40b47f0 con 0x7ff1b40cd180 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.328+0000 7ff1b0ff9700 1 -- 172.21.15.89:0/2623613920 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac007310 con 0x7ff1b40cd180 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.329+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b40b3e70 con 0x7ff1b40cd180 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.329+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b4074270 con 0x7ff1b40cd180 2023-07-06T02:23:10.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.330+0000 7ff1b0ff9700 1 -- 172.21.15.89:0/2623613920 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1ac007470 con 0x7ff1b40cd180 2023-07-06T02:23:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.330+0000 7ff1b0ff9700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff1a006ace0 0x7ff1a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.330+0000 7ff1b0ff9700 1 -- 172.21.15.89:0/2623613920 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff1ac089540 con 0x7ff1b40cd180 2023-07-06T02:23:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.330+0000 7ff1b3fff700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff1a006ace0 0x7ff1a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.331+0000 7ff1b3fff700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff1a006ace0 0x7ff1a006d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40090b0 tx=0x7ff1a4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:10.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.335+0000 7ff1b0ff9700 1 -- 172.21.15.89:0/2623613920 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff1ac054390 con 0x7ff1b40cd180 2023-07-06T02:23:10.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:10.567+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff1b40cceb0 con 0x7ff1b40cd180 2023-07-06T02:23:11.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.182+0000 7ff1b0ff9700 1 -- 172.21.15.89:0/2623613920 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff1ac0579b0 con 0x7ff1b40cd180 2023-07-06T02:23:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff1a006ace0 msgr2=0x7ff1a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff1a006ace0 0x7ff1a006d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40090b0 tx=0x7ff1a4008030 comp rx=0 tx=0).stop 2023-07-06T02:23:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40cd180 msgr2=0x7ff1b414bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40cd180 0x7ff1b414bb40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00cba0 tx=0x7ff1ac00cf60 comp rx=0 tx=0).stop 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 shutdown_connections 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff1a006ace0 0x7ff1a006d190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1b40cd180 0x7ff1b414bb40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1b406dc10 0x7ff1b40b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 --2- 172.21.15.89:0/2623613920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1b40705a0 0x7ff1b4158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 >> 172.21.15.89:0/2623613920 conn(0x7ff1b40d0b00 msgr2=0x7ff1b40d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 shutdown_connections 2023-07-06T02:23:11.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.185+0000 7ff1ba967700 1 -- 172.21.15.89:0/2623613920 wait complete. 2023-07-06T02:23:11.202 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T02:23:11.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:11.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.322+0000 7f617b65e700 1 Processor -- start 2023-07-06T02:23:11.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.323+0000 7f617b65e700 1 -- start start 2023-07-06T02:23:11.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f617b65e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f617406c5e0 0x7f61740cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f617b65e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6174160b80 0x7f61740d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f617b65e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6174161000 con 0x7f6174159fd0 2023-07-06T02:23:11.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f617b65e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61740cdb60 con 0x7f6174160b80 2023-07-06T02:23:11.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f617b65e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617415ce60 con 0x7f617406c5e0 2023-07-06T02:23:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f6178bf9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f617406c5e0 0x7f61740cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f6179bfb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6174160b80 0x7f61740d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f6178bf9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f617406c5e0 0x7f61740cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55656/0 (socket says 172.21.15.89:55656) 2023-07-06T02:23:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.324+0000 7f6178bf9700 1 -- 172.21.15.89:0/1420862002 learned_addr learned my addr 172.21.15.89:0/1420862002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:11.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f6178bf9700 1 -- 172.21.15.89:0/1420862002 >> v2:172.21.15.172:3300/0 conn(0x7f6174160b80 msgr2=0x7f61740d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:11.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f6178bf9700 1 --2- 172.21.15.89:0/1420862002 >> v2:172.21.15.172:3300/0 conn(0x7f6174160b80 0x7f61740d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f6178bf9700 1 -- 172.21.15.89:0/1420862002 >> v1:172.21.15.172:6789/0 conn(0x7f6174159fd0 legacy=0x7f617406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f6178bf9700 1 -- 172.21.15.89:0/1420862002 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617415d140 con 0x7f617406c5e0 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f6179bfb700 1 --2- 172.21.15.89:0/1420862002 >> v2:172.21.15.172:3300/0 conn(0x7f6174160b80 0x7f61740d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f6178bf9700 1 --2- 172.21.15.89:0/1420862002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f617406c5e0 0x7f61740cd4d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f61640098f0 tx=0x7f6164009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d06590fe369b085 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f616bfff700 1 -- 172.21.15.89:0/1420862002 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011830 con 0x7f617406c5e0 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f616bfff700 1 -- 172.21.15.89:0/1420862002 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6164011990 con 0x7f617406c5e0 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.325+0000 7f616bfff700 1 -- 172.21.15.89:0/1420862002 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011d20 con 0x7f617406c5e0 2023-07-06T02:23:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 -- 172.21.15.89:0/1420862002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f617406c5e0 msgr2=0x7f61740cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 --2- 172.21.15.89:0/1420862002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f617406c5e0 0x7f61740cd4d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f61640098f0 tx=0x7f6164009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 -- 172.21.15.89:0/1420862002 shutdown_connections 2023-07-06T02:23:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 --2- 172.21.15.89:0/1420862002 >> v2:172.21.15.172:3300/0 conn(0x7f6174160b80 0x7f61740d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 --2- 172.21.15.89:0/1420862002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f617406c5e0 0x7f61740cd4d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 -- 172.21.15.89:0/1420862002 >> 172.21.15.89:0/1420862002 conn(0x7f61740d0590 msgr2=0x7f61740d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 -- 172.21.15.89:0/1420862002 shutdown_connections 2023-07-06T02:23:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.326+0000 7f617b65e700 1 -- 172.21.15.89:0/1420862002 wait complete. 2023-07-06T02:23:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.327+0000 7f617b65e700 1 Processor -- start 2023-07-06T02:23:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.327+0000 7f617b65e700 1 -- start start 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f617b65e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617406c5e0 0x7f61741573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f617b65e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6174159fd0 0x7f617415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f617b65e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6174160b80 0x7f617415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f617b65e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740c64b0 con 0x7f6174160b80 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f617b65e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740cce90 con 0x7f617406c5e0 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f617b65e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61740c66c0 con 0x7f6174159fd0 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f6179bfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6174160b80 0x7f617415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f6179bfb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6174160b80 0x7f617415b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60688/0 (socket says 172.21.15.89:60688) 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f6179bfb700 1 -- 172.21.15.89:0/509845778 learned_addr learned my addr 172.21.15.89:0/509845778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f61793fa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617406c5e0 0x7f61741573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.328+0000 7f6178bf9700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6174159fd0 0x7f617415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6179bfb700 1 -- 172.21.15.89:0/509845778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6174159fd0 msgr2=0x7f617415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6179bfb700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6174159fd0 0x7f617415b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6179bfb700 1 -- 172.21.15.89:0/509845778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617406c5e0 msgr2=0x7f61741573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6179bfb700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617406c5e0 0x7f61741573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6179bfb700 1 -- 172.21.15.89:0/509845778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c009a70 con 0x7f6174160b80 2023-07-06T02:23:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6179bfb700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6174160b80 0x7f617415b950 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f616c00efc0 tx=0x7f616c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6169ffb700 1 -- 172.21.15.89:0/509845778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c00ca90 con 0x7f6174160b80 2023-07-06T02:23:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6169ffb700 1 -- 172.21.15.89:0/509845778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c00f040 con 0x7f6174160b80 2023-07-06T02:23:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6164009550 con 0x7f6174160b80 2023-07-06T02:23:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f6169ffb700 1 -- 172.21.15.89:0/509845778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c00ce30 con 0x7f6174160b80 2023-07-06T02:23:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.329+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61740c6c80 con 0x7f6174160b80 2023-07-06T02:23:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.330+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61740c6e70 con 0x7f6174160b80 2023-07-06T02:23:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.330+0000 7f6169ffb700 1 -- 172.21.15.89:0/509845778 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f616c014480 con 0x7f6174160b80 2023-07-06T02:23:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.331+0000 7f6169ffb700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f615806ace0 0x7f615806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.331+0000 7f6169ffb700 1 -- 172.21.15.89:0/509845778 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f616c089db0 con 0x7f6174160b80 2023-07-06T02:23:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.331+0000 7f61793fa700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f615806ace0 0x7f615806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.331+0000 7f61793fa700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f615806ace0 0x7f615806d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6170007fd0 tx=0x7f6170009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.335+0000 7f6169ffb700 1 -- 172.21.15.89:0/509845778 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f616c054b70 con 0x7f6174160b80 2023-07-06T02:23:11.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.568+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6174074270 con 0x7f6174160b80 2023-07-06T02:23:11.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.569+0000 7f6169ffb700 1 -- 172.21.15.89:0/509845778 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f616c018090 con 0x7f6174160b80 2023-07-06T02:23:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:11.572 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f615806ace0 msgr2=0x7f615806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f615806ace0 0x7f615806d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6170007fd0 tx=0x7f6170009040 comp rx=0 tx=0).stop 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6174160b80 msgr2=0x7f617415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6174160b80 0x7f617415b950 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f616c00efc0 tx=0x7f616c00c3a0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 shutdown_connections 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f615806ace0 0x7f615806d190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6174160b80 0x7f617415b950 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617406c5e0 0x7f61741573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 --2- 172.21.15.89:0/509845778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6174159fd0 0x7f617415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 >> 172.21.15.89:0/509845778 conn(0x7f61740d0590 msgr2=0x7f617415f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 shutdown_connections 2023-07-06T02:23:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.571+0000 7f617b65e700 1 -- 172.21.15.89:0/509845778 wait complete. 2023-07-06T02:23:11.576 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:11.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:11.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.703+0000 7f4530678700 1 Processor -- start 2023-07-06T02:23:11.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.703+0000 7f4530678700 1 -- start start 2023-07-06T02:23:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f4530678700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4528070580 0x7f4528070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f4530678700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f45280cc7f0 0x7f452806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f4530678700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f45280ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f4530678700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4528070e90 con 0x7f4528070580 2023-07-06T02:23:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f4530678700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452806b2a0 con 0x7f45280cc7f0 2023-07-06T02:23:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f4530678700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452806b580 con 0x7f45280ccd40 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4528070580 0x7f4528070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4528070580 0x7f4528070950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60692/0 (socket says 172.21.15.89:60692) 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 -- 172.21.15.89:0/1521881788 learned_addr learned my addr 172.21.15.89:0/1521881788 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452dc13700 1 --2- 172.21.15.89:0/1521881788 >> v2:172.21.15.172:3300/0 conn(0x7f45280cc7f0 0x7f452806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452ec15700 1 --2- 172.21.15.89:0/1521881788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f45280ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 -- 172.21.15.89:0/1521881788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 msgr2=0x7f45280ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 --2- 172.21.15.89:0/1521881788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f45280ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 -- 172.21.15.89:0/1521881788 >> v2:172.21.15.172:3300/0 conn(0x7f45280cc7f0 msgr2=0x7f452806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 --2- 172.21.15.89:0/1521881788 >> v2:172.21.15.172:3300/0 conn(0x7f45280cc7f0 0x7f452806c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 -- 172.21.15.89:0/1521881788 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452806b860 con 0x7f4528070580 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452e414700 1 --2- 172.21.15.89:0/1521881788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528070580 0x7f4528070950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4528070330 tx=0x7f451800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1159337da981d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452d412700 1 -- 172.21.15.89:0/1521881788 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451800dd30 con 0x7f4528070580 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452d412700 1 -- 172.21.15.89:0/1521881788 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451800de90 con 0x7f4528070580 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.705+0000 7f452d412700 1 -- 172.21.15.89:0/1521881788 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518010690 con 0x7f4528070580 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 -- 172.21.15.89:0/1521881788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528070580 msgr2=0x7f4528070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 --2- 172.21.15.89:0/1521881788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528070580 0x7f4528070950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4528070330 tx=0x7f451800be70 comp rx=0 tx=0).stop 2023-07-06T02:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 -- 172.21.15.89:0/1521881788 shutdown_connections 2023-07-06T02:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 --2- 172.21.15.89:0/1521881788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f45280ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 --2- 172.21.15.89:0/1521881788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528070580 0x7f4528070950 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 --2- 172.21.15.89:0/1521881788 >> v2:172.21.15.172:3300/0 conn(0x7f45280cc7f0 0x7f452806c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 -- 172.21.15.89:0/1521881788 >> 172.21.15.89:0/1521881788 conn(0x7f45280ce870 msgr2=0x7f45280cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 -- 172.21.15.89:0/1521881788 shutdown_connections 2023-07-06T02:23:11.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.706+0000 7f4530678700 1 -- 172.21.15.89:0/1521881788 wait complete. 2023-07-06T02:23:11.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.707+0000 7f4530678700 1 Processor -- start 2023-07-06T02:23:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.708+0000 7f4530678700 1 -- start start 2023-07-06T02:23:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.708+0000 7f4530678700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4528070580 0x7f45281f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.708+0000 7f4530678700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45280cc7f0 0x7f4528146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.708+0000 7f4530678700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f4528146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.708+0000 7f4530678700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528149740 con 0x7f45280cc7f0 2023-07-06T02:23:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.708+0000 7f4530678700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281f1810 con 0x7f4528070580 2023-07-06T02:23:11.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.708+0000 7f4530678700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45280c8840 con 0x7f45280ccd40 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45280cc7f0 0x7f4528146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45280cc7f0 0x7f4528146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60700/0 (socket says 172.21.15.89:60700) 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 -- 172.21.15.89:0/2691403921 learned_addr learned my addr 172.21.15.89:0/2691403921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452ec15700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f4528146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452e414700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4528070580 0x7f45281f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 -- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 msgr2=0x7f4528146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f4528146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 -- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4528070580 msgr2=0x7f45281f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4528070580 0x7f45281f12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 -- 172.21.15.89:0/2691403921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451800b7f0 con 0x7f45280cc7f0 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452e414700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4528070580 0x7f45281f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f452dc13700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45280cc7f0 0x7f4528146860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f452400d7d0 tx=0x7f452400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f451effd700 1 -- 172.21.15.89:0/2691403921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45240125f0 con 0x7f45280cc7f0 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f451effd700 1 -- 172.21.15.89:0/2691403921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524012750 con 0x7f45280cc7f0 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452815a880 con 0x7f45280cc7f0 2023-07-06T02:23:11.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f451effd700 1 -- 172.21.15.89:0/2691403921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524012ae0 con 0x7f45280cc7f0 2023-07-06T02:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.709+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452815ae40 con 0x7f45280cc7f0 2023-07-06T02:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.710+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452815b1b0 con 0x7f45280cc7f0 2023-07-06T02:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.711+0000 7f451effd700 1 -- 172.21.15.89:0/2691403921 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4524012c40 con 0x7f45280cc7f0 2023-07-06T02:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.711+0000 7f451effd700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f450c06ace0 0x7f450c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.711+0000 7f452e414700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f450c06ace0 0x7f450c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.711+0000 7f451effd700 1 -- 172.21.15.89:0/2691403921 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4524089ab0 con 0x7f45280cc7f0 2023-07-06T02:23:11.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.712+0000 7f452e414700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f450c06ace0 0x7f450c06d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f451800beb0 tx=0x7f4518012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:11.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.716+0000 7f451effd700 1 -- 172.21.15.89:0/2691403921 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4524054870 con 0x7f45280cc7f0 2023-07-06T02:23:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.901+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f452806e240 con 0x7f45280cc7f0 2023-07-06T02:23:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.902+0000 7f451effd700 1 -- 172.21.15.89:0/2691403921 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f4524057e90 con 0x7f45280cc7f0 2023-07-06T02:23:11.905 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:11.906 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f450c06ace0 msgr2=0x7f450c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f450c06ace0 0x7f450c06d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f451800beb0 tx=0x7f4518012000 comp rx=0 tx=0).stop 2023-07-06T02:23:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45280cc7f0 msgr2=0x7f4528146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45280cc7f0 0x7f4528146860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f452400d7d0 tx=0x7f452400db90 comp rx=0 tx=0).stop 2023-07-06T02:23:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 shutdown_connections 2023-07-06T02:23:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45280ccd40 0x7f4528146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4528070580 0x7f45281f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f450c06ace0 0x7f450c06d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 --2- 172.21.15.89:0/2691403921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45280cc7f0 0x7f4528146860 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:11.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.904+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 >> 172.21.15.89:0/2691403921 conn(0x7f45280ce870 msgr2=0x7f452806d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:11.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.905+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 shutdown_connections 2023-07-06T02:23:11.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:11.905+0000 7f4530678700 1 -- 172.21.15.89:0/2691403921 wait complete. 2023-07-06T02:23:11.920 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T02:23:11.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T02:23:12.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.037+0000 7f5aa7891700 1 Processor -- start 2023-07-06T02:23:12.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.038+0000 7f5aa7891700 1 -- start start 2023-07-06T02:23:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.038+0000 7f5aa7891700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.038+0000 7f5aa7891700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa0160bb0 0x7f5aa00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.038+0000 7f5aa7891700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5aa01600b0 0x7f5aa00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.038+0000 7f5aa7891700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa00cb3d0 con 0x7f5aa0160bb0 2023-07-06T02:23:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.038+0000 7f5aa7891700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa0156e80 con 0x7f5aa01600b0 2023-07-06T02:23:12.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.038+0000 7f5aa7891700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00ca370 con 0x7f5aa0156ab0 2023-07-06T02:23:12.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa0160bb0 0x7f5aa00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5aa0160bb0 0x7f5aa00b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60708/0 (socket says 172.21.15.89:60708) 2023-07-06T02:23:12.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/3139035759 learned_addr learned my addr 172.21.15.89:0/3139035759 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:12.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa5e2e700 1 --2- 172.21.15.89:0/3139035759 >> v2:172.21.15.172:3300/0 conn(0x7f5aa01600b0 0x7f5aa00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa562d700 1 --2- 172.21.15.89:0/3139035759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/3139035759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 msgr2=0x7f5aa0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:12.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 --2- 172.21.15.89:0/3139035759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/3139035759 >> v2:172.21.15.172:3300/0 conn(0x7f5aa01600b0 msgr2=0x7f5aa00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:12.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 --2- 172.21.15.89:0/3139035759 >> v2:172.21.15.172:3300/0 conn(0x7f5aa01600b0 0x7f5aa00b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/3139035759 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00ca610 con 0x7f5aa0160bb0 2023-07-06T02:23:12.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa5e2e700 1 --2- 172.21.15.89:0/3139035759 >> v2:172.21.15.172:3300/0 conn(0x7f5aa01600b0 0x7f5aa00b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:12.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5aa4e2c700 1 --2- 172.21.15.89:0/3139035759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa0160bb0 0x7f5aa00b1490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5aa015fef0 tx=0x7f5a9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5584ea77bc688d02 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5a97fff700 1 -- 172.21.15.89:0/3139035759 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00b870 con 0x7f5aa0160bb0 2023-07-06T02:23:12.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5a97fff700 1 -- 172.21.15.89:0/3139035759 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c00b9d0 con 0x7f5aa0160bb0 2023-07-06T02:23:12.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.039+0000 7f5a97fff700 1 -- 172.21.15.89:0/3139035759 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00bd60 con 0x7f5aa0160bb0 2023-07-06T02:23:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 -- 172.21.15.89:0/3139035759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa0160bb0 msgr2=0x7f5aa00b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 --2- 172.21.15.89:0/3139035759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa0160bb0 0x7f5aa00b1490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5aa015fef0 tx=0x7f5a9c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 -- 172.21.15.89:0/3139035759 shutdown_connections 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 --2- 172.21.15.89:0/3139035759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 --2- 172.21.15.89:0/3139035759 >> v2:172.21.15.172:3300/0 conn(0x7f5aa01600b0 0x7f5aa00b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 --2- 172.21.15.89:0/3139035759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa0160bb0 0x7f5aa00b1490 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 -- 172.21.15.89:0/3139035759 >> 172.21.15.89:0/3139035759 conn(0x7f5aa00cc490 msgr2=0x7f5aa00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 -- 172.21.15.89:0/3139035759 shutdown_connections 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.040+0000 7f5aa7891700 1 -- 172.21.15.89:0/3139035759 wait complete. 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.041+0000 7f5aa7891700 1 Processor -- start 2023-07-06T02:23:12.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.041+0000 7f5aa7891700 1 -- start start 2023-07-06T02:23:12.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa7891700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa7891700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa01600b0 0x7f5aa00b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa7891700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aa00b4460 0x7f5aa00b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa7891700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00b5780 con 0x7f5aa01600b0 2023-07-06T02:23:12.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa7891700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00b51c0 con 0x7f5aa00b4460 2023-07-06T02:23:12.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa7891700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00b54a0 con 0x7f5aa0156ab0 2023-07-06T02:23:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa4e2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa01600b0 0x7f5aa00b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa4e2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa01600b0 0x7f5aa00b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60714/0 (socket says 172.21.15.89:60714) 2023-07-06T02:23:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.042+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/2432603840 learned_addr learned my addr 172.21.15.89:0/2432603840 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa562d700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa5e2e700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aa00b4460 0x7f5aa00b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 msgr2=0x7f5aa0154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa4e2c700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aa00b4460 msgr2=0x7f5aa00b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa4e2c700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aa00b4460 0x7f5aa00b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa4e2c700 1 -- 172.21.15.89:0/2432603840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c009550 con 0x7f5aa01600b0 2023-07-06T02:23:12.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa562d700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:12.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa5e2e700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aa00b4460 0x7f5aa00b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:12.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa4e2c700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa01600b0 0x7f5aa00b2860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c000c00 tx=0x7f5a9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2432603840 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004520 con 0x7f5aa01600b0 2023-07-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2432603840 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c01e070 con 0x7f5aa01600b0 2023-07-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2432603840 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0038c0 con 0x7f5aa01600b0 2023-07-06T02:23:12.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa00aca70 con 0x7f5aa01600b0 2023-07-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.043+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa00c3540 con 0x7f5aa01600b0 2023-07-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.044+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa00acf90 con 0x7f5aa01600b0 2023-07-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.044+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2432603840 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a9c003a20 con 0x7f5aa01600b0 2023-07-06T02:23:12.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.045+0000 7f5a95ffb700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5a8406ace0 0x7f5a8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.045+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2432603840 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5a9c020070 con 0x7f5aa01600b0 2023-07-06T02:23:12.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.045+0000 7f5aa562d700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5a8406ace0 0x7f5a8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.045+0000 7f5aa562d700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5a8406ace0 0x7f5a8406d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009a00 tx=0x7f5a90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.049+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2432603840 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5aa00acf90 con 0x7f5aa01600b0 2023-07-06T02:23:12.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.237+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 --> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5aa00ca8f0 con 0x7f5a8406ace0 2023-07-06T02:23:12.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.248+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2432603840 <== mgr.4100 v2:172.21.15.89:6832/3591115711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367553 (secure 0 0 0) 0x7f5a90009610 con 0x7f5a8406ace0 2023-07-06T02:23:12.255 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:12.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5a8406ace0 msgr2=0x7f5a8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5a8406ace0 0x7f5a8406d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009a00 tx=0x7f5a90008040 comp rx=0 tx=0).stop 2023-07-06T02:23:12.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa01600b0 msgr2=0x7f5aa00b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa01600b0 0x7f5aa00b2860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c000c00 tx=0x7f5a9c004750 comp rx=0 tx=0).stop 2023-07-06T02:23:12.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 shutdown_connections 2023-07-06T02:23:12.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aa00b4460 0x7f5aa00b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5aa0156ab0 0x7f5aa0154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5a8406ace0 0x7f5a8406d190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 --2- 172.21.15.89:0/2432603840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5aa01600b0 0x7f5aa00b2860 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 >> 172.21.15.89:0/2432603840 conn(0x7f5aa00cc490 msgr2=0x7f5aa00b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:12.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 shutdown_connections 2023-07-06T02:23:12.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.251+0000 7f5aa7891700 1 -- 172.21.15.89:0/2432603840 wait complete. 2023-07-06T02:23:12.261 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-07-06T02:23:12.267 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-07-06T02:23:11.981525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218820,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755900,"statfs":{"total":773094113280,"available":772870041600,"internally_reserved":0,"allocated":3280896,"data_stored":2026399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002677"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462978+0000","last_change":"2023-07-06T02:22:53.433054+0000","last_active":"2023-07-06T02:22:58.462978+0000","last_peered":"2023-07-06T02:22:58.462978+0000","last_clean":"2023-07-06T02:22:58.462978+0000","last_became_active":"2023-07-06T02:22:53.432749+0000","last_became_peered":"2023-07-06T02:22:53.432749+0000","last_unstale":"2023-07-06T02:22:58.462978+0000","last_undegraded":"2023-07-06T02:22:58.462978+0000","last_fullsized":"2023-07-06T02:22:58.462978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:42.597214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643097+0000","last_change":"2023-07-06T02:22:53.425731+0000","last_active":"2023-07-06T02:22:59.643097+0000","last_peered":"2023-07-06T02:22:59.643097+0000","last_clean":"2023-07-06T02:22:59.643097+0000","last_became_active":"2023-07-06T02:22:53.425403+0000","last_became_peered":"2023-07-06T02:22:53.425403+0000","last_unstale":"2023-07-06T02:22:59.643097+0000","last_undegraded":"2023-07-06T02:22:59.643097+0000","last_fullsized":"2023-07-06T02:22:59.643097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:00.409252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623835+0000","last_change":"2023-07-06T02:22:53.413131+0000","last_active":"2023-07-06T02:22:58.623835+0000","last_peered":"2023-07-06T02:22:58.623835+0000","last_clean":"2023-07-06T02:22:58.623835+0000","last_became_active":"2023-07-06T02:22:53.412839+0000","last_became_peered":"2023-07-06T02:22:53.412839+0000","last_unstale":"2023-07-06T02:22:58.623835+0000","last_undegraded":"2023-07-06T02:22:58.623835+0000","last_fullsized":"2023-07-06T02:22:58.623835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:51.080641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463166+0000","last_change":"2023-07-06T02:22:53.420056+0000","last_active":"2023-07-06T02:22:58.463166+0000","last_peered":"2023-07-06T02:22:58.463166+0000","last_clean":"2023-07-06T02:22:58.463166+0000","last_became_active":"2023-07-06T02:22:53.419779+0000","last_became_peered":"2023-07-06T02:22:53.419779+0000","last_unstale":"2023-07-06T02:22:58.463166+0000","last_undegraded":"2023-07-06T02:22:58.463166+0000","last_fullsized":"2023-07-06T02:22:58.463166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:37.102968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642705+0000","last_change":"2023-07-06T02:22:53.420867+0000","last_active":"2023-07-06T02:22:59.642705+0000","last_peered":"2023-07-06T02:22:59.642705+0000","last_clean":"2023-07-06T02:22:59.642705+0000","last_became_active":"2023-07-06T02:22:53.419716+0000","last_became_peered":"2023-07-06T02:22:53.419716+0000","last_unstale":"2023-07-06T02:22:59.642705+0000","last_undegraded":"2023-07-06T02:22:59.642705+0000","last_fullsized":"2023-07-06T02:22:59.642705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:33.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623263+0000","last_change":"2023-07-06T02:22:53.412521+0000","last_active":"2023-07-06T02:22:58.623263+0000","last_peered":"2023-07-06T02:22:58.623263+0000","last_clean":"2023-07-06T02:22:58.623263+0000","last_became_active":"2023-07-06T02:22:53.412230+0000","last_became_peered":"2023-07-06T02:22:53.412230+0000","last_unstale":"2023-07-06T02:22:58.623263+0000","last_undegraded":"2023-07-06T02:22:58.623263+0000","last_fullsized":"2023-07-06T02:22:58.623263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:44.353524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545634+0000","last_change":"2023-07-06T02:22:53.427954+0000","last_active":"2023-07-06T02:22:58.545634+0000","last_peered":"2023-07-06T02:22:58.545634+0000","last_clean":"2023-07-06T02:22:58.545634+0000","last_became_active":"2023-07-06T02:22:53.426315+0000","last_became_peered":"2023-07-06T02:22:53.426315+0000","last_unstale":"2023-07-06T02:22:58.545634+0000","last_undegraded":"2023-07-06T02:22:58.545634+0000","last_fullsized":"2023-07-06T02:22:58.545634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642615+0000","last_change":"2023-07-06T02:22:53.425882+0000","last_active":"2023-07-06T02:22:59.642615+0000","last_peered":"2023-07-06T02:22:59.642615+0000","last_clean":"2023-07-06T02:22:59.642615+0000","last_became_active":"2023-07-06T02:22:53.425583+0000","last_became_peered":"2023-07-06T02:22:53.425583+0000","last_unstale":"2023-07-06T02:22:59.642615+0000","last_undegraded":"2023-07-06T02:22:59.642615+0000","last_fullsized":"2023-07-06T02:22:59.642615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:51.387481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642268+0000","last_change":"2023-07-06T02:22:53.427106+0000","last_active":"2023-07-06T02:22:59.642268+0000","last_peered":"2023-07-06T02:22:59.642268+0000","last_clean":"2023-07-06T02:22:59.642268+0000","last_became_active":"2023-07-06T02:22:53.426852+0000","last_became_peered":"2023-07-06T02:22:53.426852+0000","last_unstale":"2023-07-06T02:22:59.642268+0000","last_undegraded":"2023-07-06T02:22:59.642268+0000","last_fullsized":"2023-07-06T02:22:59.642268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643507+0000","last_change":"2023-07-06T02:22:53.421592+0000","last_active":"2023-07-06T02:22:59.643507+0000","last_peered":"2023-07-06T02:22:59.643507+0000","last_clean":"2023-07-06T02:22:59.643507+0000","last_became_active":"2023-07-06T02:22:53.421245+0000","last_became_peered":"2023-07-06T02:22:53.421245+0000","last_unstale":"2023-07-06T02:22:59.643507+0000","last_undegraded":"2023-07-06T02:22:59.643507+0000","last_fullsized":"2023-07-06T02:22:59.643507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623746+0000","last_change":"2023-07-06T02:22:53.424792+0000","last_active":"2023-07-06T02:22:58.623746+0000","last_peered":"2023-07-06T02:22:58.623746+0000","last_clean":"2023-07-06T02:22:58.623746+0000","last_became_active":"2023-07-06T02:22:53.424498+0000","last_became_peered":"2023-07-06T02:22:53.424498+0000","last_unstale":"2023-07-06T02:22:58.623746+0000","last_undegraded":"2023-07-06T02:22:58.623746+0000","last_fullsized":"2023-07-06T02:22:58.623746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:55:40.483266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462649+0000","last_change":"2023-07-06T02:22:53.424739+0000","last_active":"2023-07-06T02:22:58.462649+0000","last_peered":"2023-07-06T02:22:58.462649+0000","last_clean":"2023-07-06T02:22:58.462649+0000","last_became_active":"2023-07-06T02:22:53.424382+0000","last_became_peered":"2023-07-06T02:22:53.424382+0000","last_unstale":"2023-07-06T02:22:58.462649+0000","last_undegraded":"2023-07-06T02:22:58.462649+0000","last_fullsized":"2023-07-06T02:22:58.462649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:08:50.637616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462913+0000","last_change":"2023-07-06T02:22:53.435357+0000","last_active":"2023-07-06T02:22:58.462913+0000","last_peered":"2023-07-06T02:22:58.462913+0000","last_clean":"2023-07-06T02:22:58.462913+0000","last_became_active":"2023-07-06T02:22:53.435068+0000","last_became_peered":"2023-07-06T02:22:53.435068+0000","last_unstale":"2023-07-06T02:22:58.462913+0000","last_undegraded":"2023-07-06T02:22:58.462913+0000","last_fullsized":"2023-07-06T02:22:58.462913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:05.764438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642840+0000","last_change":"2023-07-06T02:22:53.419307+0000","last_active":"2023-07-06T02:22:59.642840+0000","last_peered":"2023-07-06T02:22:59.642840+0000","last_clean":"2023-07-06T02:22:59.642840+0000","last_became_active":"2023-07-06T02:22:53.419010+0000","last_became_peered":"2023-07-06T02:22:53.419010+0000","last_unstale":"2023-07-06T02:22:59.642840+0000","last_undegraded":"2023-07-06T02:22:59.642840+0000","last_fullsized":"2023-07-06T02:22:59.642840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:52.740513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545431+0000","last_change":"2023-07-06T02:22:53.427932+0000","last_active":"2023-07-06T02:22:58.545431+0000","last_peered":"2023-07-06T02:22:58.545431+0000","last_clean":"2023-07-06T02:22:58.545431+0000","last_became_active":"2023-07-06T02:22:53.426280+0000","last_became_peered":"2023-07-06T02:22:53.426280+0000","last_unstale":"2023-07-06T02:22:58.545431+0000","last_undegraded":"2023-07-06T02:22:58.545431+0000","last_fullsized":"2023-07-06T02:22:58.545431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:05.507588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622966+0000","last_change":"2023-07-06T02:22:53.412012+0000","last_active":"2023-07-06T02:22:58.622966+0000","last_peered":"2023-07-06T02:22:58.622966+0000","last_clean":"2023-07-06T02:22:58.622966+0000","last_became_active":"2023-07-06T02:22:53.411675+0000","last_became_peered":"2023-07-06T02:22:53.411675+0000","last_unstale":"2023-07-06T02:22:58.622966+0000","last_undegraded":"2023-07-06T02:22:58.622966+0000","last_fullsized":"2023-07-06T02:22:58.622966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:13:54.565351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:53.432526+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:53.432240+0000","last_became_peered":"2023-07-06T02:22:53.432240+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623666+0000","last_change":"2023-07-06T02:22:53.419854+0000","last_active":"2023-07-06T02:22:58.623666+0000","last_peered":"2023-07-06T02:22:58.623666+0000","last_clean":"2023-07-06T02:22:58.623666+0000","last_became_active":"2023-07-06T02:22:53.419569+0000","last_became_peered":"2023-07-06T02:22:53.419569+0000","last_unstale":"2023-07-06T02:22:58.623666+0000","last_undegraded":"2023-07-06T02:22:58.623666+0000","last_fullsized":"2023-07-06T02:22:58.623666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:44:47.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463157+0000","last_change":"2023-07-06T02:22:53.422202+0000","last_active":"2023-07-06T02:22:58.463157+0000","last_peered":"2023-07-06T02:22:58.463157+0000","last_clean":"2023-07-06T02:22:58.463157+0000","last_became_active":"2023-07-06T02:22:53.421981+0000","last_became_peered":"2023-07-06T02:22:53.421981+0000","last_unstale":"2023-07-06T02:22:58.463157+0000","last_undegraded":"2023-07-06T02:22:58.463157+0000","last_fullsized":"2023-07-06T02:22:58.463157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:55.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623252+0000","last_change":"2023-07-06T02:22:53.421294+0000","last_active":"2023-07-06T02:22:58.623252+0000","last_peered":"2023-07-06T02:22:58.623252+0000","last_clean":"2023-07-06T02:22:58.623252+0000","last_became_active":"2023-07-06T02:22:53.420987+0000","last_became_peered":"2023-07-06T02:22:53.420987+0000","last_unstale":"2023-07-06T02:22:58.623252+0000","last_undegraded":"2023-07-06T02:22:58.623252+0000","last_fullsized":"2023-07-06T02:22:58.623252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:07:23.523735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545995+0000","last_change":"2023-07-06T02:22:53.423456+0000","last_active":"2023-07-06T02:22:58.545995+0000","last_peered":"2023-07-06T02:22:58.545995+0000","last_clean":"2023-07-06T02:22:58.545995+0000","last_became_active":"2023-07-06T02:22:53.423131+0000","last_became_peered":"2023-07-06T02:22:53.423131+0000","last_unstale":"2023-07-06T02:22:58.545995+0000","last_undegraded":"2023-07-06T02:22:58.545995+0000","last_fullsized":"2023-07-06T02:22:58.545995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641976+0000","last_change":"2023-07-06T02:22:53.424047+0000","last_active":"2023-07-06T02:22:59.641976+0000","last_peered":"2023-07-06T02:22:59.641976+0000","last_clean":"2023-07-06T02:22:59.641976+0000","last_became_active":"2023-07-06T02:22:53.423692+0000","last_became_peered":"2023-07-06T02:22:53.423692+0000","last_unstale":"2023-07-06T02:22:59.641976+0000","last_undegraded":"2023-07-06T02:22:59.641976+0000","last_fullsized":"2023-07-06T02:22:59.641976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.432531+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.432245+0000","last_became_peered":"2023-07-06T02:22:53.432245+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:09:56.578120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545462+0000","last_change":"2023-07-06T02:22:53.426016+0000","last_active":"2023-07-06T02:22:58.545462+0000","last_peered":"2023-07-06T02:22:58.545462+0000","last_clean":"2023-07-06T02:22:58.545462+0000","last_became_active":"2023-07-06T02:22:53.417030+0000","last_became_peered":"2023-07-06T02:22:53.417030+0000","last_unstale":"2023-07-06T02:22:58.545462+0000","last_undegraded":"2023-07-06T02:22:58.545462+0000","last_fullsized":"2023-07-06T02:22:58.545462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497209+0000","last_change":"2023-07-06T02:22:53.422907+0000","last_active":"2023-07-06T02:22:58.497209+0000","last_peered":"2023-07-06T02:22:58.497209+0000","last_clean":"2023-07-06T02:22:58.497209+0000","last_became_active":"2023-07-06T02:22:53.422617+0000","last_became_peered":"2023-07-06T02:22:53.422617+0000","last_unstale":"2023-07-06T02:22:58.497209+0000","last_undegraded":"2023-07-06T02:22:58.497209+0000","last_fullsized":"2023-07-06T02:22:58.497209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:41.517576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497094+0000","last_change":"2023-07-06T02:22:53.422413+0000","last_active":"2023-07-06T02:22:58.497094+0000","last_peered":"2023-07-06T02:22:58.497094+0000","last_clean":"2023-07-06T02:22:58.497094+0000","last_became_active":"2023-07-06T02:22:53.422025+0000","last_became_peered":"2023-07-06T02:22:53.422025+0000","last_unstale":"2023-07-06T02:22:58.497094+0000","last_undegraded":"2023-07-06T02:22:58.497094+0000","last_fullsized":"2023-07-06T02:22:58.497094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:23.425050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643229+0000","last_change":"2023-07-06T02:22:53.434688+0000","last_active":"2023-07-06T02:22:59.643229+0000","last_peered":"2023-07-06T02:22:59.643229+0000","last_clean":"2023-07-06T02:22:59.643229+0000","last_became_active":"2023-07-06T02:22:53.434423+0000","last_became_peered":"2023-07-06T02:22:53.434423+0000","last_unstale":"2023-07-06T02:22:59.643229+0000","last_undegraded":"2023-07-06T02:22:59.643229+0000","last_fullsized":"2023-07-06T02:22:59.643229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:11.004232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623583+0000","last_change":"2023-07-06T02:22:57.463283+0000","last_active":"2023-07-06T02:22:58.623583+0000","last_peered":"2023-07-06T02:22:58.623583+0000","last_clean":"2023-07-06T02:22:58.623583+0000","last_became_active":"2023-07-06T02:22:57.462913+0000","last_became_peered":"2023-07-06T02:22:57.462913+0000","last_unstale":"2023-07-06T02:22:58.623583+0000","last_undegraded":"2023-07-06T02:22:58.623583+0000","last_fullsized":"2023-07-06T02:22:58.623583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:29.980455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623061+0000","last_change":"2023-07-06T02:22:52.423689+0000","last_active":"2023-07-06T02:22:58.623061+0000","last_peered":"2023-07-06T02:22:58.623061+0000","last_clean":"2023-07-06T02:22:58.623061+0000","last_became_active":"2023-07-06T02:22:52.423464+0000","last_became_peered":"2023-07-06T02:22:52.423464+0000","last_unstale":"2023-07-06T02:22:58.623061+0000","last_undegraded":"2023-07-06T02:22:58.623061+0000","last_fullsized":"2023-07-06T02:22:58.623061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:34.698538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463653+0000","last_change":"2023-07-06T02:22:53.426231+0000","last_active":"2023-07-06T02:22:58.463653+0000","last_peered":"2023-07-06T02:22:58.463653+0000","last_clean":"2023-07-06T02:22:58.463653+0000","last_became_active":"2023-07-06T02:22:53.425923+0000","last_became_peered":"2023-07-06T02:22:53.425923+0000","last_unstale":"2023-07-06T02:22:58.463653+0000","last_undegraded":"2023-07-06T02:22:58.463653+0000","last_fullsized":"2023-07-06T02:22:58.463653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:22.845352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463216+0000","last_change":"2023-07-06T02:22:57.461661+0000","last_active":"2023-07-06T02:22:58.463216+0000","last_peered":"2023-07-06T02:22:58.463216+0000","last_clean":"2023-07-06T02:22:58.463216+0000","last_became_active":"2023-07-06T02:22:57.461262+0000","last_became_peered":"2023-07-06T02:22:57.461262+0000","last_unstale":"2023-07-06T02:22:58.463216+0000","last_undegraded":"2023-07-06T02:22:58.463216+0000","last_fullsized":"2023-07-06T02:22:58.463216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:05.846802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496610+0000","last_change":"2023-07-06T02:22:52.424261+0000","last_active":"2023-07-06T02:22:58.496610+0000","last_peered":"2023-07-06T02:22:58.496610+0000","last_clean":"2023-07-06T02:22:58.496610+0000","last_became_active":"2023-07-06T02:22:52.423984+0000","last_became_peered":"2023-07-06T02:22:52.423984+0000","last_unstale":"2023-07-06T02:22:58.496610+0000","last_undegraded":"2023-07-06T02:22:58.496610+0000","last_fullsized":"2023-07-06T02:22:58.496610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:56.520485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642313+0000","last_change":"2023-07-06T02:22:53.417163+0000","last_active":"2023-07-06T02:22:59.642313+0000","last_peered":"2023-07-06T02:22:59.642313+0000","last_clean":"2023-07-06T02:22:59.642313+0000","last_became_active":"2023-07-06T02:22:53.416875+0000","last_became_peered":"2023-07-06T02:22:53.416875+0000","last_unstale":"2023-07-06T02:22:59.642313+0000","last_undegraded":"2023-07-06T02:22:59.642313+0000","last_fullsized":"2023-07-06T02:22:59.642313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642441+0000","last_change":"2023-07-06T02:22:57.455750+0000","last_active":"2023-07-06T02:22:59.642441+0000","last_peered":"2023-07-06T02:22:59.642441+0000","last_clean":"2023-07-06T02:22:59.642441+0000","last_became_active":"2023-07-06T02:22:57.455331+0000","last_became_peered":"2023-07-06T02:22:57.455331+0000","last_unstale":"2023-07-06T02:22:59.642441+0000","last_undegraded":"2023-07-06T02:22:59.642441+0000","last_fullsized":"2023-07-06T02:22:59.642441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462903+0000","last_change":"2023-07-06T02:22:52.416577+0000","last_active":"2023-07-06T02:22:58.462903+0000","last_peered":"2023-07-06T02:22:58.462903+0000","last_clean":"2023-07-06T02:22:58.462903+0000","last_became_active":"2023-07-06T02:22:52.416144+0000","last_became_peered":"2023-07-06T02:22:52.416144+0000","last_unstale":"2023-07-06T02:22:58.462903+0000","last_undegraded":"2023-07-06T02:22:58.462903+0000","last_fullsized":"2023-07-06T02:22:58.462903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:03:22.897984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463486+0000","last_change":"2023-07-06T02:22:53.429089+0000","last_active":"2023-07-06T02:22:58.463486+0000","last_peered":"2023-07-06T02:22:58.463486+0000","last_clean":"2023-07-06T02:22:58.463486+0000","last_became_active":"2023-07-06T02:22:53.428809+0000","last_became_peered":"2023-07-06T02:22:53.428809+0000","last_unstale":"2023-07-06T02:22:58.463486+0000","last_undegraded":"2023-07-06T02:22:58.463486+0000","last_fullsized":"2023-07-06T02:22:58.463486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:30:16.217797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462711+0000","last_change":"2023-07-06T02:22:57.455092+0000","last_active":"2023-07-06T02:22:58.462711+0000","last_peered":"2023-07-06T02:22:58.462711+0000","last_clean":"2023-07-06T02:22:58.462711+0000","last_became_active":"2023-07-06T02:22:57.454707+0000","last_became_peered":"2023-07-06T02:22:57.454707+0000","last_unstale":"2023-07-06T02:22:58.462711+0000","last_undegraded":"2023-07-06T02:22:58.462711+0000","last_fullsized":"2023-07-06T02:22:58.462711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:09:16.226734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:52.428104+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:52.427863+0000","last_became_peered":"2023-07-06T02:22:52.427863+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:52.064862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.424797+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.424474+0000","last_became_peered":"2023-07-06T02:22:53.424474+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:31:13.843201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545356+0000","last_change":"2023-07-06T02:22:57.457309+0000","last_active":"2023-07-06T02:22:58.545356+0000","last_peered":"2023-07-06T02:22:58.545356+0000","last_clean":"2023-07-06T02:22:58.545356+0000","last_became_active":"2023-07-06T02:22:57.457008+0000","last_became_peered":"2023-07-06T02:22:57.457008+0000","last_unstale":"2023-07-06T02:22:58.545356+0000","last_undegraded":"2023-07-06T02:22:58.545356+0000","last_fullsized":"2023-07-06T02:22:58.545356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:04.437413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643326+0000","last_change":"2023-07-06T02:22:52.423048+0000","last_active":"2023-07-06T02:22:59.643326+0000","last_peered":"2023-07-06T02:22:59.643326+0000","last_clean":"2023-07-06T02:22:59.643326+0000","last_became_active":"2023-07-06T02:22:52.422801+0000","last_became_peered":"2023-07-06T02:22:52.422801+0000","last_unstale":"2023-07-06T02:22:59.643326+0000","last_undegraded":"2023-07-06T02:22:59.643326+0000","last_fullsized":"2023-07-06T02:22:59.643326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:29.807668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462478+0000","last_change":"2023-07-06T02:22:53.434004+0000","last_active":"2023-07-06T02:22:58.462478+0000","last_peered":"2023-07-06T02:22:58.462478+0000","last_clean":"2023-07-06T02:22:58.462478+0000","last_became_active":"2023-07-06T02:22:53.433739+0000","last_became_peered":"2023-07-06T02:22:53.433739+0000","last_unstale":"2023-07-06T02:22:58.462478+0000","last_undegraded":"2023-07-06T02:22:58.462478+0000","last_fullsized":"2023-07-06T02:22:58.462478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463415+0000","last_change":"2023-07-06T02:22:57.452160+0000","last_active":"2023-07-06T02:22:58.463415+0000","last_peered":"2023-07-06T02:22:58.463415+0000","last_clean":"2023-07-06T02:22:58.463415+0000","last_became_active":"2023-07-06T02:22:57.451507+0000","last_became_peered":"2023-07-06T02:22:57.451507+0000","last_unstale":"2023-07-06T02:22:58.463415+0000","last_undegraded":"2023-07-06T02:22:58.463415+0000","last_fullsized":"2023-07-06T02:22:58.463415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:10:18.060753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546029+0000","last_change":"2023-07-06T02:22:52.419096+0000","last_active":"2023-07-06T02:22:58.546029+0000","last_peered":"2023-07-06T02:22:58.546029+0000","last_clean":"2023-07-06T02:22:58.546029+0000","last_became_active":"2023-07-06T02:22:52.418844+0000","last_became_peered":"2023-07-06T02:22:52.418844+0000","last_unstale":"2023-07-06T02:22:58.546029+0000","last_undegraded":"2023-07-06T02:22:58.546029+0000","last_fullsized":"2023-07-06T02:22:58.546029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:14.974840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496797+0000","last_change":"2023-07-06T02:22:53.425547+0000","last_active":"2023-07-06T02:22:58.496797+0000","last_peered":"2023-07-06T02:22:58.496797+0000","last_clean":"2023-07-06T02:22:58.496797+0000","last_became_active":"2023-07-06T02:22:53.425239+0000","last_became_peered":"2023-07-06T02:22:53.425239+0000","last_unstale":"2023-07-06T02:22:58.496797+0000","last_undegraded":"2023-07-06T02:22:58.496797+0000","last_fullsized":"2023-07-06T02:22:58.496797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:32.244431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496944+0000","last_change":"2023-07-06T02:22:57.453836+0000","last_active":"2023-07-06T02:22:58.496944+0000","last_peered":"2023-07-06T02:22:58.496944+0000","last_clean":"2023-07-06T02:22:58.496944+0000","last_became_active":"2023-07-06T02:22:57.451637+0000","last_became_peered":"2023-07-06T02:22:57.451637+0000","last_unstale":"2023-07-06T02:22:58.496944+0000","last_undegraded":"2023-07-06T02:22:58.496944+0000","last_fullsized":"2023-07-06T02:22:58.496944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:45.805424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:23:03.321857+0000","last_change":"2023-07-06T02:22:52.421021+0000","last_active":"2023-07-06T02:23:03.321857+0000","last_peered":"2023-07-06T02:23:03.321857+0000","last_clean":"2023-07-06T02:23:03.321857+0000","last_became_active":"2023-07-06T02:22:52.420768+0000","last_became_peered":"2023-07-06T02:22:52.420768+0000","last_unstale":"2023-07-06T02:23:03.321857+0000","last_undegraded":"2023-07-06T02:23:03.321857+0000","last_fullsized":"2023-07-06T02:23:03.321857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:10:06.879321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623414+0000","last_change":"2023-07-06T02:22:53.420729+0000","last_active":"2023-07-06T02:22:58.623414+0000","last_peered":"2023-07-06T02:22:58.623414+0000","last_clean":"2023-07-06T02:22:58.623414+0000","last_became_active":"2023-07-06T02:22:53.418685+0000","last_became_peered":"2023-07-06T02:22:53.418685+0000","last_unstale":"2023-07-06T02:22:58.623414+0000","last_undegraded":"2023-07-06T02:22:58.623414+0000","last_fullsized":"2023-07-06T02:22:58.623414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:47.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643458+0000","last_change":"2023-07-06T02:22:57.455310+0000","last_active":"2023-07-06T02:22:59.643458+0000","last_peered":"2023-07-06T02:22:59.643458+0000","last_clean":"2023-07-06T02:22:59.643458+0000","last_became_active":"2023-07-06T02:22:57.454518+0000","last_became_peered":"2023-07-06T02:22:57.454518+0000","last_unstale":"2023-07-06T02:22:59.643458+0000","last_undegraded":"2023-07-06T02:22:59.643458+0000","last_fullsized":"2023-07-06T02:22:59.643458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:57.100343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545849+0000","last_change":"2023-07-06T02:22:52.421399+0000","last_active":"2023-07-06T02:22:58.545849+0000","last_peered":"2023-07-06T02:22:58.545849+0000","last_clean":"2023-07-06T02:22:58.545849+0000","last_became_active":"2023-07-06T02:22:52.420062+0000","last_became_peered":"2023-07-06T02:22:52.420062+0000","last_unstale":"2023-07-06T02:22:58.545849+0000","last_undegraded":"2023-07-06T02:22:58.545849+0000","last_fullsized":"2023-07-06T02:22:58.545849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:53.425567+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:53.425272+0000","last_became_peered":"2023-07-06T02:22:53.425272+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:45.959463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463853+0000","last_change":"2023-07-06T02:22:57.452727+0000","last_active":"2023-07-06T02:22:58.463853+0000","last_peered":"2023-07-06T02:22:58.463853+0000","last_clean":"2023-07-06T02:22:58.463853+0000","last_became_active":"2023-07-06T02:22:57.452043+0000","last_became_peered":"2023-07-06T02:22:57.452043+0000","last_unstale":"2023-07-06T02:22:58.463853+0000","last_undegraded":"2023-07-06T02:22:58.463853+0000","last_fullsized":"2023-07-06T02:22:58.463853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:26.074085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642190+0000","last_change":"2023-07-06T02:22:52.420802+0000","last_active":"2023-07-06T02:22:59.642190+0000","last_peered":"2023-07-06T02:22:59.642190+0000","last_clean":"2023-07-06T02:22:59.642190+0000","last_became_active":"2023-07-06T02:22:52.420494+0000","last_became_peered":"2023-07-06T02:22:52.420494+0000","last_unstale":"2023-07-06T02:22:59.642190+0000","last_undegraded":"2023-07-06T02:22:59.642190+0000","last_fullsized":"2023-07-06T02:22:59.642190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:04.500268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643228+0000","last_change":"2023-07-06T02:22:53.432233+0000","last_active":"2023-07-06T02:22:59.643228+0000","last_peered":"2023-07-06T02:22:59.643228+0000","last_clean":"2023-07-06T02:22:59.643228+0000","last_became_active":"2023-07-06T02:22:53.431893+0000","last_became_peered":"2023-07-06T02:22:53.431893+0000","last_unstale":"2023-07-06T02:22:59.643228+0000","last_undegraded":"2023-07-06T02:22:59.643228+0000","last_fullsized":"2023-07-06T02:22:59.643228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463256+0000","last_change":"2023-07-06T02:22:57.455213+0000","last_active":"2023-07-06T02:22:58.463256+0000","last_peered":"2023-07-06T02:22:58.463256+0000","last_clean":"2023-07-06T02:22:58.463256+0000","last_became_active":"2023-07-06T02:22:57.454800+0000","last_became_peered":"2023-07-06T02:22:57.454800+0000","last_unstale":"2023-07-06T02:22:58.463256+0000","last_undegraded":"2023-07-06T02:22:58.463256+0000","last_fullsized":"2023-07-06T02:22:58.463256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:00.449720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545483+0000","last_change":"2023-07-06T02:22:52.421475+0000","last_active":"2023-07-06T02:22:58.545483+0000","last_peered":"2023-07-06T02:22:58.545483+0000","last_clean":"2023-07-06T02:22:58.545483+0000","last_became_active":"2023-07-06T02:22:52.421267+0000","last_became_peered":"2023-07-06T02:22:52.421267+0000","last_unstale":"2023-07-06T02:22:58.545483+0000","last_undegraded":"2023-07-06T02:22:58.545483+0000","last_fullsized":"2023-07-06T02:22:58.545483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:09.996912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642587+0000","last_change":"2023-07-06T02:22:53.420060+0000","last_active":"2023-07-06T02:22:59.642587+0000","last_peered":"2023-07-06T02:22:59.642587+0000","last_clean":"2023-07-06T02:22:59.642587+0000","last_became_active":"2023-07-06T02:22:53.419347+0000","last_became_peered":"2023-07-06T02:22:53.419347+0000","last_unstale":"2023-07-06T02:22:59.642587+0000","last_undegraded":"2023-07-06T02:22:59.642587+0000","last_fullsized":"2023-07-06T02:22:59.642587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643160+0000","last_change":"2023-07-06T02:22:57.453231+0000","last_active":"2023-07-06T02:22:59.643160+0000","last_peered":"2023-07-06T02:22:59.643160+0000","last_clean":"2023-07-06T02:22:59.643160+0000","last_became_active":"2023-07-06T02:22:57.452776+0000","last_became_peered":"2023-07-06T02:22:57.452776+0000","last_unstale":"2023-07-06T02:22:59.643160+0000","last_undegraded":"2023-07-06T02:22:59.643160+0000","last_fullsized":"2023-07-06T02:22:59.643160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:09.535749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462794+0000","last_change":"2023-07-06T02:22:52.417281+0000","last_active":"2023-07-06T02:22:58.462794+0000","last_peered":"2023-07-06T02:22:58.462794+0000","last_clean":"2023-07-06T02:22:58.462794+0000","last_became_active":"2023-07-06T02:22:52.417011+0000","last_became_peered":"2023-07-06T02:22:52.417011+0000","last_unstale":"2023-07-06T02:22:58.462794+0000","last_undegraded":"2023-07-06T02:22:58.462794+0000","last_fullsized":"2023-07-06T02:22:58.462794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:56.885245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463676+0000","last_change":"2023-07-06T02:22:53.433973+0000","last_active":"2023-07-06T02:22:58.463676+0000","last_peered":"2023-07-06T02:22:58.463676+0000","last_clean":"2023-07-06T02:22:58.463676+0000","last_became_active":"2023-07-06T02:22:53.433561+0000","last_became_peered":"2023-07-06T02:22:53.433561+0000","last_unstale":"2023-07-06T02:22:58.463676+0000","last_undegraded":"2023-07-06T02:22:58.463676+0000","last_fullsized":"2023-07-06T02:22:58.463676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:36.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545710+0000","last_change":"2023-07-06T02:22:57.455016+0000","last_active":"2023-07-06T02:22:58.545710+0000","last_peered":"2023-07-06T02:22:58.545710+0000","last_clean":"2023-07-06T02:22:58.545710+0000","last_became_active":"2023-07-06T02:22:57.454627+0000","last_became_peered":"2023-07-06T02:22:57.454627+0000","last_unstale":"2023-07-06T02:22:58.545710+0000","last_undegraded":"2023-07-06T02:22:58.545710+0000","last_fullsized":"2023-07-06T02:22:58.545710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546232+0000","last_change":"2023-07-06T02:22:52.420410+0000","last_active":"2023-07-06T02:22:58.546232+0000","last_peered":"2023-07-06T02:22:58.546232+0000","last_clean":"2023-07-06T02:22:58.546232+0000","last_became_active":"2023-07-06T02:22:52.420089+0000","last_became_peered":"2023-07-06T02:22:52.420089+0000","last_unstale":"2023-07-06T02:22:58.546232+0000","last_undegraded":"2023-07-06T02:22:58.546232+0000","last_fullsized":"2023-07-06T02:22:58.546232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642490+0000","last_change":"2023-07-06T02:22:53.432339+0000","last_active":"2023-07-06T02:22:59.642490+0000","last_peered":"2023-07-06T02:22:59.642490+0000","last_clean":"2023-07-06T02:22:59.642490+0000","last_became_active":"2023-07-06T02:22:53.431993+0000","last_became_peered":"2023-07-06T02:22:53.431993+0000","last_unstale":"2023-07-06T02:22:59.642490+0000","last_undegraded":"2023-07-06T02:22:59.642490+0000","last_fullsized":"2023-07-06T02:22:59.642490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462370+0000","last_change":"2023-07-06T02:22:57.457792+0000","last_active":"2023-07-06T02:22:58.462370+0000","last_peered":"2023-07-06T02:22:58.462370+0000","last_clean":"2023-07-06T02:22:58.462370+0000","last_became_active":"2023-07-06T02:22:57.457396+0000","last_became_peered":"2023-07-06T02:22:57.457396+0000","last_unstale":"2023-07-06T02:22:58.462370+0000","last_undegraded":"2023-07-06T02:22:58.462370+0000","last_fullsized":"2023-07-06T02:22:58.462370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:17.206072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546155+0000","last_change":"2023-07-06T02:22:52.421892+0000","last_active":"2023-07-06T02:22:58.546155+0000","last_peered":"2023-07-06T02:22:58.546155+0000","last_clean":"2023-07-06T02:22:58.546155+0000","last_became_active":"2023-07-06T02:22:52.421613+0000","last_became_peered":"2023-07-06T02:22:52.421613+0000","last_unstale":"2023-07-06T02:22:58.546155+0000","last_undegraded":"2023-07-06T02:22:58.546155+0000","last_fullsized":"2023-07-06T02:22:58.546155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622924+0000","last_change":"2023-07-06T02:22:53.422219+0000","last_active":"2023-07-06T02:22:58.622924+0000","last_peered":"2023-07-06T02:22:58.622924+0000","last_clean":"2023-07-06T02:22:58.622924+0000","last_became_active":"2023-07-06T02:22:53.421934+0000","last_became_peered":"2023-07-06T02:22:53.421934+0000","last_unstale":"2023-07-06T02:22:58.622924+0000","last_undegraded":"2023-07-06T02:22:58.622924+0000","last_fullsized":"2023-07-06T02:22:58.622924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:14:41.243585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462824+0000","last_change":"2023-07-06T02:22:57.450611+0000","last_active":"2023-07-06T02:22:58.462824+0000","last_peered":"2023-07-06T02:22:58.462824+0000","last_clean":"2023-07-06T02:22:58.462824+0000","last_became_active":"2023-07-06T02:22:57.450255+0000","last_became_peered":"2023-07-06T02:22:57.450255+0000","last_unstale":"2023-07-06T02:22:58.462824+0000","last_undegraded":"2023-07-06T02:22:58.462824+0000","last_fullsized":"2023-07-06T02:22:58.462824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:04.461198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545892+0000","last_change":"2023-07-06T02:22:52.420450+0000","last_active":"2023-07-06T02:22:58.545892+0000","last_peered":"2023-07-06T02:22:58.545892+0000","last_clean":"2023-07-06T02:22:58.545892+0000","last_became_active":"2023-07-06T02:22:52.420171+0000","last_became_peered":"2023-07-06T02:22:52.420171+0000","last_unstale":"2023-07-06T02:22:58.545892+0000","last_undegraded":"2023-07-06T02:22:58.545892+0000","last_fullsized":"2023-07-06T02:22:58.545892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:35.849912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462735+0000","last_change":"2023-07-06T02:22:53.425285+0000","last_active":"2023-07-06T02:22:58.462735+0000","last_peered":"2023-07-06T02:22:58.462735+0000","last_clean":"2023-07-06T02:22:58.462735+0000","last_became_active":"2023-07-06T02:22:53.424994+0000","last_became_peered":"2023-07-06T02:22:53.424994+0000","last_unstale":"2023-07-06T02:22:58.462735+0000","last_undegraded":"2023-07-06T02:22:58.462735+0000","last_fullsized":"2023-07-06T02:22:58.462735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.580209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462875+0000","last_change":"2023-07-06T02:22:57.459471+0000","last_active":"2023-07-06T02:22:58.462875+0000","last_peered":"2023-07-06T02:22:58.462875+0000","last_clean":"2023-07-06T02:22:58.462875+0000","last_became_active":"2023-07-06T02:22:57.459086+0000","last_became_peered":"2023-07-06T02:22:57.459086+0000","last_unstale":"2023-07-06T02:22:58.462875+0000","last_undegraded":"2023-07-06T02:22:58.462875+0000","last_fullsized":"2023-07-06T02:22:58.462875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:32.995367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643074+0000","last_change":"2023-07-06T02:22:52.419379+0000","last_active":"2023-07-06T02:22:59.643074+0000","last_peered":"2023-07-06T02:22:59.643074+0000","last_clean":"2023-07-06T02:22:59.643074+0000","last_became_active":"2023-07-06T02:22:52.419100+0000","last_became_peered":"2023-07-06T02:22:52.419100+0000","last_unstale":"2023-07-06T02:22:59.643074+0000","last_undegraded":"2023-07-06T02:22:59.643074+0000","last_fullsized":"2023-07-06T02:22:59.643074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:15.612393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642483+0000","last_change":"2023-07-06T02:22:53.423846+0000","last_active":"2023-07-06T02:22:59.642483+0000","last_peered":"2023-07-06T02:22:59.642483+0000","last_clean":"2023-07-06T02:22:59.642483+0000","last_became_active":"2023-07-06T02:22:53.423585+0000","last_became_peered":"2023-07-06T02:22:53.423585+0000","last_unstale":"2023-07-06T02:22:59.642483+0000","last_undegraded":"2023-07-06T02:22:59.642483+0000","last_fullsized":"2023-07-06T02:22:59.642483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:05.040887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462786+0000","last_change":"2023-07-06T02:22:57.450230+0000","last_active":"2023-07-06T02:22:58.462786+0000","last_peered":"2023-07-06T02:22:58.462786+0000","last_clean":"2023-07-06T02:22:58.462786+0000","last_became_active":"2023-07-06T02:22:57.449841+0000","last_became_peered":"2023-07-06T02:22:57.449841+0000","last_unstale":"2023-07-06T02:22:58.462786+0000","last_undegraded":"2023-07-06T02:22:58.462786+0000","last_fullsized":"2023-07-06T02:22:58.462786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:26:07.959918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462612+0000","last_change":"2023-07-06T02:22:52.418001+0000","last_active":"2023-07-06T02:22:58.462612+0000","last_peered":"2023-07-06T02:22:58.462612+0000","last_clean":"2023-07-06T02:22:58.462612+0000","last_became_active":"2023-07-06T02:22:52.417737+0000","last_became_peered":"2023-07-06T02:22:52.417737+0000","last_unstale":"2023-07-06T02:22:58.462612+0000","last_undegraded":"2023-07-06T02:22:58.462612+0000","last_fullsized":"2023-07-06T02:22:58.462612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623068+0000","last_change":"2023-07-06T02:22:53.418072+0000","last_active":"2023-07-06T02:22:58.623068+0000","last_peered":"2023-07-06T02:22:58.623068+0000","last_clean":"2023-07-06T02:22:58.623068+0000","last_became_active":"2023-07-06T02:22:53.416070+0000","last_became_peered":"2023-07-06T02:22:53.416070+0000","last_unstale":"2023-07-06T02:22:58.623068+0000","last_undegraded":"2023-07-06T02:22:58.623068+0000","last_fullsized":"2023-07-06T02:22:58.623068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:39:55.241653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497345+0000","last_change":"2023-07-06T02:22:57.451350+0000","last_active":"2023-07-06T02:22:58.497345+0000","last_peered":"2023-07-06T02:22:58.497345+0000","last_clean":"2023-07-06T02:22:58.497345+0000","last_became_active":"2023-07-06T02:22:57.450988+0000","last_became_peered":"2023-07-06T02:22:57.450988+0000","last_unstale":"2023-07-06T02:22:58.497345+0000","last_undegraded":"2023-07-06T02:22:58.497345+0000","last_fullsized":"2023-07-06T02:22:58.497345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:08:02.280780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462907+0000","last_change":"2023-07-06T02:22:52.423191+0000","last_active":"2023-07-06T02:22:58.462907+0000","last_peered":"2023-07-06T02:22:58.462907+0000","last_clean":"2023-07-06T02:22:58.462907+0000","last_became_active":"2023-07-06T02:22:52.422863+0000","last_became_peered":"2023-07-06T02:22:52.422863+0000","last_unstale":"2023-07-06T02:22:58.462907+0000","last_undegraded":"2023-07-06T02:22:58.462907+0000","last_fullsized":"2023-07-06T02:22:58.462907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:22.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623436+0000","last_change":"2023-07-06T02:22:53.419350+0000","last_active":"2023-07-06T02:22:58.623436+0000","last_peered":"2023-07-06T02:22:58.623436+0000","last_clean":"2023-07-06T02:22:58.623436+0000","last_became_active":"2023-07-06T02:22:53.419024+0000","last_became_peered":"2023-07-06T02:22:53.419024+0000","last_unstale":"2023-07-06T02:22:58.623436+0000","last_undegraded":"2023-07-06T02:22:58.623436+0000","last_fullsized":"2023-07-06T02:22:58.623436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:33.746001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643344+0000","last_change":"2023-07-06T02:22:57.455212+0000","last_active":"2023-07-06T02:22:59.643344+0000","last_peered":"2023-07-06T02:22:59.643344+0000","last_clean":"2023-07-06T02:22:59.643344+0000","last_became_active":"2023-07-06T02:22:57.454773+0000","last_became_peered":"2023-07-06T02:22:57.454773+0000","last_unstale":"2023-07-06T02:22:59.643344+0000","last_undegraded":"2023-07-06T02:22:59.643344+0000","last_fullsized":"2023-07-06T02:22:59.643344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622698+0000","last_change":"2023-07-06T02:22:52.417326+0000","last_active":"2023-07-06T02:22:58.622698+0000","last_peered":"2023-07-06T02:22:58.622698+0000","last_clean":"2023-07-06T02:22:58.622698+0000","last_became_active":"2023-07-06T02:22:52.417065+0000","last_became_peered":"2023-07-06T02:22:52.417065+0000","last_unstale":"2023-07-06T02:22:58.622698+0000","last_undegraded":"2023-07-06T02:22:58.622698+0000","last_fullsized":"2023-07-06T02:22:58.622698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:35.235033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623747+0000","last_change":"2023-07-06T02:22:53.422511+0000","last_active":"2023-07-06T02:22:58.623747+0000","last_peered":"2023-07-06T02:22:58.623747+0000","last_clean":"2023-07-06T02:22:58.623747+0000","last_became_active":"2023-07-06T02:22:53.420266+0000","last_became_peered":"2023-07-06T02:22:53.420266+0000","last_unstale":"2023-07-06T02:22:58.623747+0000","last_undegraded":"2023-07-06T02:22:58.623747+0000","last_fullsized":"2023-07-06T02:22:58.623747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:05.329078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623647+0000","last_change":"2023-07-06T02:22:57.451156+0000","last_active":"2023-07-06T02:22:58.623647+0000","last_peered":"2023-07-06T02:22:58.623647+0000","last_clean":"2023-07-06T02:22:58.623647+0000","last_became_active":"2023-07-06T02:22:57.450784+0000","last_became_peered":"2023-07-06T02:22:57.450784+0000","last_unstale":"2023-07-06T02:22:58.623647+0000","last_undegraded":"2023-07-06T02:22:58.623647+0000","last_fullsized":"2023-07-06T02:22:58.623647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:15:13.002024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462668+0000","last_change":"2023-07-06T02:22:52.418727+0000","last_active":"2023-07-06T02:22:58.462668+0000","last_peered":"2023-07-06T02:22:58.462668+0000","last_clean":"2023-07-06T02:22:58.462668+0000","last_became_active":"2023-07-06T02:22:52.418454+0000","last_became_peered":"2023-07-06T02:22:52.418454+0000","last_unstale":"2023-07-06T02:22:58.462668+0000","last_undegraded":"2023-07-06T02:22:58.462668+0000","last_fullsized":"2023-07-06T02:22:58.462668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:05.425463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497348+0000","last_change":"2023-07-06T02:22:53.432617+0000","last_active":"2023-07-06T02:22:58.497348+0000","last_peered":"2023-07-06T02:22:58.497348+0000","last_clean":"2023-07-06T02:22:58.497348+0000","last_became_active":"2023-07-06T02:22:53.432332+0000","last_became_peered":"2023-07-06T02:22:53.432332+0000","last_unstale":"2023-07-06T02:22:58.497348+0000","last_undegraded":"2023-07-06T02:22:58.497348+0000","last_fullsized":"2023-07-06T02:22:58.497348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623091+0000","last_change":"2023-07-06T02:22:57.463354+0000","last_active":"2023-07-06T02:22:58.623091+0000","last_peered":"2023-07-06T02:22:58.623091+0000","last_clean":"2023-07-06T02:22:58.623091+0000","last_became_active":"2023-07-06T02:22:57.462934+0000","last_became_peered":"2023-07-06T02:22:57.462934+0000","last_unstale":"2023-07-06T02:22:58.623091+0000","last_undegraded":"2023-07-06T02:22:58.623091+0000","last_fullsized":"2023-07-06T02:22:58.623091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:23.690834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463354+0000","last_change":"2023-07-06T02:22:52.421219+0000","last_active":"2023-07-06T02:22:58.463354+0000","last_peered":"2023-07-06T02:22:58.463354+0000","last_clean":"2023-07-06T02:22:58.463354+0000","last_became_active":"2023-07-06T02:22:52.420927+0000","last_became_peered":"2023-07-06T02:22:52.420927+0000","last_unstale":"2023-07-06T02:22:58.463354+0000","last_undegraded":"2023-07-06T02:22:58.463354+0000","last_fullsized":"2023-07-06T02:22:58.463354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:17:11.011220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623582+0000","last_change":"2023-07-06T02:22:47.973696+0000","last_active":"2023-07-06T02:22:58.623582+0000","last_peered":"2023-07-06T02:22:58.623582+0000","last_clean":"2023-07-06T02:22:58.623582+0000","last_became_active":"2023-07-06T02:22:47.973358+0000","last_became_peered":"2023-07-06T02:22:47.973358+0000","last_unstale":"2023-07-06T02:22:58.623582+0000","last_undegraded":"2023-07-06T02:22:58.623582+0000","last_fullsized":"2023-07-06T02:22:58.623582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:26:55.060583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462551+0000","last_change":"2023-07-06T02:22:53.433182+0000","last_active":"2023-07-06T02:22:58.462551+0000","last_peered":"2023-07-06T02:22:58.462551+0000","last_clean":"2023-07-06T02:22:58.462551+0000","last_became_active":"2023-07-06T02:22:53.432900+0000","last_became_peered":"2023-07-06T02:22:53.432900+0000","last_unstale":"2023-07-06T02:22:58.462551+0000","last_undegraded":"2023-07-06T02:22:58.462551+0000","last_fullsized":"2023-07-06T02:22:58.462551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462417+0000","last_change":"2023-07-06T02:22:57.456985+0000","last_active":"2023-07-06T02:22:58.462417+0000","last_peered":"2023-07-06T02:22:58.462417+0000","last_clean":"2023-07-06T02:22:58.462417+0000","last_became_active":"2023-07-06T02:22:57.456594+0000","last_became_peered":"2023-07-06T02:22:57.456594+0000","last_unstale":"2023-07-06T02:22:58.462417+0000","last_undegraded":"2023-07-06T02:22:58.462417+0000","last_fullsized":"2023-07-06T02:22:58.462417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:44.241938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463270+0000","last_change":"2023-07-06T02:22:47.323237+0000","last_active":"2023-07-06T02:22:58.463270+0000","last_peered":"2023-07-06T02:22:58.463270+0000","last_clean":"2023-07-06T02:22:58.463270+0000","last_became_active":"2023-07-06T02:22:47.322763+0000","last_became_peered":"2023-07-06T02:22:47.322763+0000","last_unstale":"2023-07-06T02:22:58.463270+0000","last_undegraded":"2023-07-06T02:22:58.463270+0000","last_fullsized":"2023-07-06T02:22:58.463270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:51:10.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642236+0000","last_change":"2023-07-06T02:22:52.417097+0000","last_active":"2023-07-06T02:22:59.642236+0000","last_peered":"2023-07-06T02:22:59.642236+0000","last_clean":"2023-07-06T02:22:59.642236+0000","last_became_active":"2023-07-06T02:22:52.416824+0000","last_became_peered":"2023-07-06T02:22:52.416824+0000","last_unstale":"2023-07-06T02:22:59.642236+0000","last_undegraded":"2023-07-06T02:22:59.642236+0000","last_fullsized":"2023-07-06T02:22:59.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:35.415046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622712+0000","last_change":"2023-07-06T02:22:53.419559+0000","last_active":"2023-07-06T02:22:58.622712+0000","last_peered":"2023-07-06T02:22:58.622712+0000","last_clean":"2023-07-06T02:22:58.622712+0000","last_became_active":"2023-07-06T02:22:53.419069+0000","last_became_peered":"2023-07-06T02:22:53.419069+0000","last_unstale":"2023-07-06T02:22:58.622712+0000","last_undegraded":"2023-07-06T02:22:58.622712+0000","last_fullsized":"2023-07-06T02:22:58.622712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:22.045924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643230+0000","last_change":"2023-07-06T02:22:57.459369+0000","last_active":"2023-07-06T02:22:59.643230+0000","last_peered":"2023-07-06T02:22:59.643230+0000","last_clean":"2023-07-06T02:22:59.643230+0000","last_became_active":"2023-07-06T02:22:57.458965+0000","last_became_peered":"2023-07-06T02:22:57.458965+0000","last_unstale":"2023-07-06T02:22:59.643230+0000","last_undegraded":"2023-07-06T02:22:59.643230+0000","last_fullsized":"2023-07-06T02:22:59.643230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:40.450527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496831+0000","last_change":"2023-07-06T02:22:47.325598+0000","last_active":"2023-07-06T02:22:58.496831+0000","last_peered":"2023-07-06T02:22:58.496831+0000","last_clean":"2023-07-06T02:22:58.496831+0000","last_became_active":"2023-07-06T02:22:47.325315+0000","last_became_peered":"2023-07-06T02:22:47.325315+0000","last_unstale":"2023-07-06T02:22:58.496831+0000","last_undegraded":"2023-07-06T02:22:58.496831+0000","last_fullsized":"2023-07-06T02:22:58.496831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:36.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545612+0000","last_change":"2023-07-06T02:22:47.326882+0000","last_active":"2023-07-06T02:22:58.545612+0000","last_peered":"2023-07-06T02:22:58.545612+0000","last_clean":"2023-07-06T02:22:58.545612+0000","last_became_active":"2023-07-06T02:22:47.326511+0000","last_became_peered":"2023-07-06T02:22:47.326511+0000","last_unstale":"2023-07-06T02:22:58.545612+0000","last_undegraded":"2023-07-06T02:22:58.545612+0000","last_fullsized":"2023-07-06T02:22:58.545612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:46.985907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641990+0000","last_change":"2023-07-06T02:22:52.416206+0000","last_active":"2023-07-06T02:22:59.641990+0000","last_peered":"2023-07-06T02:22:59.641990+0000","last_clean":"2023-07-06T02:22:59.641990+0000","last_became_active":"2023-07-06T02:22:52.415905+0000","last_became_peered":"2023-07-06T02:22:52.415905+0000","last_unstale":"2023-07-06T02:22:59.641990+0000","last_undegraded":"2023-07-06T02:22:59.641990+0000","last_fullsized":"2023-07-06T02:22:59.641990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:59.401932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642535+0000","last_change":"2023-07-06T02:22:53.423988+0000","last_active":"2023-07-06T02:22:59.642535+0000","last_peered":"2023-07-06T02:22:59.642535+0000","last_clean":"2023-07-06T02:22:59.642535+0000","last_became_active":"2023-07-06T02:22:53.423711+0000","last_became_peered":"2023-07-06T02:22:53.423711+0000","last_unstale":"2023-07-06T02:22:59.642535+0000","last_undegraded":"2023-07-06T02:22:59.642535+0000","last_fullsized":"2023-07-06T02:22:59.642535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463130+0000","last_change":"2023-07-06T02:22:57.453513+0000","last_active":"2023-07-06T02:22:58.463130+0000","last_peered":"2023-07-06T02:22:58.463130+0000","last_clean":"2023-07-06T02:22:58.463130+0000","last_became_active":"2023-07-06T02:22:57.453163+0000","last_became_peered":"2023-07-06T02:22:57.453163+0000","last_unstale":"2023-07-06T02:22:58.463130+0000","last_undegraded":"2023-07-06T02:22:58.463130+0000","last_fullsized":"2023-07-06T02:22:58.463130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:06:26.324856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545616+0000","last_change":"2023-07-06T02:22:47.613219+0000","last_active":"2023-07-06T02:22:58.545616+0000","last_peered":"2023-07-06T02:22:58.545616+0000","last_clean":"2023-07-06T02:22:58.545616+0000","last_became_active":"2023-07-06T02:22:47.612787+0000","last_became_peered":"2023-07-06T02:22:47.612787+0000","last_unstale":"2023-07-06T02:22:58.545616+0000","last_undegraded":"2023-07-06T02:22:58.545616+0000","last_fullsized":"2023-07-06T02:22:58.545616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:24.728713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496749+0000","last_change":"2023-07-06T02:22:52.421146+0000","last_active":"2023-07-06T02:22:58.496749+0000","last_peered":"2023-07-06T02:22:58.496749+0000","last_clean":"2023-07-06T02:22:58.496749+0000","last_became_active":"2023-07-06T02:22:52.420937+0000","last_became_peered":"2023-07-06T02:22:52.420937+0000","last_unstale":"2023-07-06T02:22:58.496749+0000","last_undegraded":"2023-07-06T02:22:58.496749+0000","last_fullsized":"2023-07-06T02:22:58.496749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:41.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497242+0000","last_change":"2023-07-06T02:22:53.428966+0000","last_active":"2023-07-06T02:22:58.497242+0000","last_peered":"2023-07-06T02:22:58.497242+0000","last_clean":"2023-07-06T02:22:58.497242+0000","last_became_active":"2023-07-06T02:22:53.428644+0000","last_became_peered":"2023-07-06T02:22:53.428644+0000","last_unstale":"2023-07-06T02:22:58.497242+0000","last_undegraded":"2023-07-06T02:22:58.497242+0000","last_fullsized":"2023-07-06T02:22:58.497242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:10.006953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545738+0000","last_change":"2023-07-06T02:22:57.445443+0000","last_active":"2023-07-06T02:22:58.545738+0000","last_peered":"2023-07-06T02:22:58.545738+0000","last_clean":"2023-07-06T02:22:58.545738+0000","last_became_active":"2023-07-06T02:22:57.445042+0000","last_became_peered":"2023-07-06T02:22:57.445042+0000","last_unstale":"2023-07-06T02:22:58.545738+0000","last_undegraded":"2023-07-06T02:22:58.545738+0000","last_fullsized":"2023-07-06T02:22:58.545738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:09.069048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545450+0000","last_change":"2023-07-06T02:22:47.613102+0000","last_active":"2023-07-06T02:22:58.545450+0000","last_peered":"2023-07-06T02:22:58.545450+0000","last_clean":"2023-07-06T02:22:58.545450+0000","last_became_active":"2023-07-06T02:22:47.612655+0000","last_became_peered":"2023-07-06T02:22:47.612655+0000","last_unstale":"2023-07-06T02:22:58.545450+0000","last_undegraded":"2023-07-06T02:22:58.545450+0000","last_fullsized":"2023-07-06T02:22:58.545450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:30.336091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462609+0000","last_change":"2023-07-06T02:22:52.419398+0000","last_active":"2023-07-06T02:22:58.462609+0000","last_peered":"2023-07-06T02:22:58.462609+0000","last_clean":"2023-07-06T02:22:58.462609+0000","last_became_active":"2023-07-06T02:22:52.419139+0000","last_became_peered":"2023-07-06T02:22:52.419139+0000","last_unstale":"2023-07-06T02:22:58.462609+0000","last_undegraded":"2023-07-06T02:22:58.462609+0000","last_fullsized":"2023-07-06T02:22:58.462609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463313+0000","last_change":"2023-07-06T02:22:53.425795+0000","last_active":"2023-07-06T02:22:58.463313+0000","last_peered":"2023-07-06T02:22:58.463313+0000","last_clean":"2023-07-06T02:22:58.463313+0000","last_became_active":"2023-07-06T02:22:53.425518+0000","last_became_peered":"2023-07-06T02:22:53.425518+0000","last_unstale":"2023-07-06T02:22:58.463313+0000","last_undegraded":"2023-07-06T02:22:58.463313+0000","last_fullsized":"2023-07-06T02:22:58.463313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:21:40.141734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642407+0000","last_change":"2023-07-06T02:22:57.457031+0000","last_active":"2023-07-06T02:22:59.642407+0000","last_peered":"2023-07-06T02:22:59.642407+0000","last_clean":"2023-07-06T02:22:59.642407+0000","last_became_active":"2023-07-06T02:22:57.456653+0000","last_became_peered":"2023-07-06T02:22:57.456653+0000","last_unstale":"2023-07-06T02:22:59.642407+0000","last_undegraded":"2023-07-06T02:22:59.642407+0000","last_fullsized":"2023-07-06T02:22:59.642407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:34.843149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642558+0000","last_change":"2023-07-06T02:22:47.613009+0000","last_active":"2023-07-06T02:22:59.642558+0000","last_peered":"2023-07-06T02:22:59.642558+0000","last_clean":"2023-07-06T02:22:59.642558+0000","last_became_active":"2023-07-06T02:22:47.612664+0000","last_became_peered":"2023-07-06T02:22:47.612664+0000","last_unstale":"2023-07-06T02:22:59.642558+0000","last_undegraded":"2023-07-06T02:22:59.642558+0000","last_fullsized":"2023-07-06T02:22:59.642558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:35.690988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623032+0000","last_change":"2023-07-06T02:22:52.417974+0000","last_active":"2023-07-06T02:22:58.623032+0000","last_peered":"2023-07-06T02:22:58.623032+0000","last_clean":"2023-07-06T02:22:58.623032+0000","last_became_active":"2023-07-06T02:22:52.417681+0000","last_became_peered":"2023-07-06T02:22:52.417681+0000","last_unstale":"2023-07-06T02:22:58.623032+0000","last_undegraded":"2023-07-06T02:22:58.623032+0000","last_fullsized":"2023-07-06T02:22:58.623032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:28:34.626494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496776+0000","last_change":"2023-07-06T02:22:53.424117+0000","last_active":"2023-07-06T02:22:58.496776+0000","last_peered":"2023-07-06T02:22:58.496776+0000","last_clean":"2023-07-06T02:22:58.496776+0000","last_became_active":"2023-07-06T02:22:53.423755+0000","last_became_peered":"2023-07-06T02:22:53.423755+0000","last_unstale":"2023-07-06T02:22:58.496776+0000","last_undegraded":"2023-07-06T02:22:58.496776+0000","last_fullsized":"2023-07-06T02:22:58.496776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:48.677462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463117+0000","last_change":"2023-07-06T02:22:57.451130+0000","last_active":"2023-07-06T02:22:58.463117+0000","last_peered":"2023-07-06T02:22:58.463117+0000","last_clean":"2023-07-06T02:22:58.463117+0000","last_became_active":"2023-07-06T02:22:57.450726+0000","last_became_peered":"2023-07-06T02:22:57.450726+0000","last_unstale":"2023-07-06T02:22:58.463117+0000","last_undegraded":"2023-07-06T02:22:58.463117+0000","last_fullsized":"2023-07-06T02:22:58.463117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:38.492011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463258+0000","last_change":"2023-07-06T02:22:47.612949+0000","last_active":"2023-07-06T02:22:58.463258+0000","last_peered":"2023-07-06T02:22:58.463258+0000","last_clean":"2023-07-06T02:22:58.463258+0000","last_became_active":"2023-07-06T02:22:47.612606+0000","last_became_peered":"2023-07-06T02:22:47.612606+0000","last_unstale":"2023-07-06T02:22:58.463258+0000","last_undegraded":"2023-07-06T02:22:58.463258+0000","last_fullsized":"2023-07-06T02:22:58.463258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:26:58.278350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496991+0000","last_change":"2023-07-06T02:22:52.421787+0000","last_active":"2023-07-06T02:22:58.496991+0000","last_peered":"2023-07-06T02:22:58.496991+0000","last_clean":"2023-07-06T02:22:58.496991+0000","last_became_active":"2023-07-06T02:22:52.421536+0000","last_became_peered":"2023-07-06T02:22:52.421536+0000","last_unstale":"2023-07-06T02:22:58.496991+0000","last_undegraded":"2023-07-06T02:22:58.496991+0000","last_fullsized":"2023-07-06T02:22:58.496991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:50.380161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642624+0000","last_change":"2023-07-06T02:22:53.419643+0000","last_active":"2023-07-06T02:22:59.642624+0000","last_peered":"2023-07-06T02:22:59.642624+0000","last_clean":"2023-07-06T02:22:59.642624+0000","last_became_active":"2023-07-06T02:22:53.418153+0000","last_became_peered":"2023-07-06T02:22:53.418153+0000","last_unstale":"2023-07-06T02:22:59.642624+0000","last_undegraded":"2023-07-06T02:22:59.642624+0000","last_fullsized":"2023-07-06T02:22:59.642624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642319+0000","last_change":"2023-07-06T02:22:57.455969+0000","last_active":"2023-07-06T02:22:59.642319+0000","last_peered":"2023-07-06T02:22:59.642319+0000","last_clean":"2023-07-06T02:22:59.642319+0000","last_became_active":"2023-07-06T02:22:57.455566+0000","last_became_peered":"2023-07-06T02:22:57.455566+0000","last_unstale":"2023-07-06T02:22:59.642319+0000","last_undegraded":"2023-07-06T02:22:59.642319+0000","last_fullsized":"2023-07-06T02:22:59.642319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:15.032393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463409+0000","last_change":"2023-07-06T02:22:47.329340+0000","last_active":"2023-07-06T02:22:58.463409+0000","last_peered":"2023-07-06T02:22:58.463409+0000","last_clean":"2023-07-06T02:22:58.463409+0000","last_became_active":"2023-07-06T02:22:47.329090+0000","last_became_peered":"2023-07-06T02:22:47.329090+0000","last_unstale":"2023-07-06T02:22:58.463409+0000","last_undegraded":"2023-07-06T02:22:58.463409+0000","last_fullsized":"2023-07-06T02:22:58.463409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:42.929148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643052+0000","last_change":"2023-07-06T02:22:52.422406+0000","last_active":"2023-07-06T02:22:59.643052+0000","last_peered":"2023-07-06T02:22:59.643052+0000","last_clean":"2023-07-06T02:22:59.643052+0000","last_became_active":"2023-07-06T02:22:52.422125+0000","last_became_peered":"2023-07-06T02:22:52.422125+0000","last_unstale":"2023-07-06T02:22:59.643052+0000","last_undegraded":"2023-07-06T02:22:59.643052+0000","last_fullsized":"2023-07-06T02:22:59.643052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:08.999634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496575+0000","last_change":"2023-07-06T02:22:53.428897+0000","last_active":"2023-07-06T02:22:58.496575+0000","last_peered":"2023-07-06T02:22:58.496575+0000","last_clean":"2023-07-06T02:22:58.496575+0000","last_became_active":"2023-07-06T02:22:53.428651+0000","last_became_peered":"2023-07-06T02:22:53.428651+0000","last_unstale":"2023-07-06T02:22:58.496575+0000","last_undegraded":"2023-07-06T02:22:58.496575+0000","last_fullsized":"2023-07-06T02:22:58.496575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:49.714737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497099+0000","last_change":"2023-07-06T02:22:57.454674+0000","last_active":"2023-07-06T02:22:58.497099+0000","last_peered":"2023-07-06T02:22:58.497099+0000","last_clean":"2023-07-06T02:22:58.497099+0000","last_became_active":"2023-07-06T02:22:57.454162+0000","last_became_peered":"2023-07-06T02:22:57.454162+0000","last_unstale":"2023-07-06T02:22:58.497099+0000","last_undegraded":"2023-07-06T02:22:58.497099+0000","last_fullsized":"2023-07-06T02:22:58.497099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:56.768674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642632+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:59.642632+0000","last_peered":"2023-07-06T02:22:59.642632+0000","last_clean":"2023-07-06T02:22:59.642632+0000","last_became_active":"2023-07-06T02:22:52.418794+0000","last_became_peered":"2023-07-06T02:22:52.418794+0000","last_unstale":"2023-07-06T02:22:59.642632+0000","last_undegraded":"2023-07-06T02:22:59.642632+0000","last_fullsized":"2023-07-06T02:22:59.642632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463505+0000","last_change":"2023-07-06T02:22:53.424846+0000","last_active":"2023-07-06T02:22:58.463505+0000","last_peered":"2023-07-06T02:22:58.463505+0000","last_clean":"2023-07-06T02:22:58.463505+0000","last_became_active":"2023-07-06T02:22:53.424606+0000","last_became_peered":"2023-07-06T02:22:53.424606+0000","last_unstale":"2023-07-06T02:22:58.463505+0000","last_undegraded":"2023-07-06T02:22:58.463505+0000","last_fullsized":"2023-07-06T02:22:58.463505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:31:39.592702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545637+0000","last_change":"2023-07-06T02:22:57.455280+0000","last_active":"2023-07-06T02:22:58.545637+0000","last_peered":"2023-07-06T02:22:58.545637+0000","last_clean":"2023-07-06T02:22:58.545637+0000","last_became_active":"2023-07-06T02:22:57.454860+0000","last_became_peered":"2023-07-06T02:22:57.454860+0000","last_unstale":"2023-07-06T02:22:58.545637+0000","last_undegraded":"2023-07-06T02:22:58.545637+0000","last_fullsized":"2023-07-06T02:22:58.545637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642892+0000","last_change":"2023-07-06T02:22:52.419322+0000","last_active":"2023-07-06T02:22:59.642892+0000","last_peered":"2023-07-06T02:22:59.642892+0000","last_clean":"2023-07-06T02:22:59.642892+0000","last_became_active":"2023-07-06T02:22:52.418950+0000","last_became_peered":"2023-07-06T02:22:52.418950+0000","last_unstale":"2023-07-06T02:22:59.642892+0000","last_undegraded":"2023-07-06T02:22:59.642892+0000","last_fullsized":"2023-07-06T02:22:59.642892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:22.417730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623010+0000","last_change":"2023-07-06T02:22:53.419784+0000","last_active":"2023-07-06T02:22:58.623010+0000","last_peered":"2023-07-06T02:22:58.623010+0000","last_clean":"2023-07-06T02:22:58.623010+0000","last_became_active":"2023-07-06T02:22:53.419477+0000","last_became_peered":"2023-07-06T02:22:53.419477+0000","last_unstale":"2023-07-06T02:22:58.623010+0000","last_undegraded":"2023-07-06T02:22:58.623010+0000","last_fullsized":"2023-07-06T02:22:58.623010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:07.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643049+0000","last_change":"2023-07-06T02:22:57.447138+0000","last_active":"2023-07-06T02:22:59.643049+0000","last_peered":"2023-07-06T02:22:59.643049+0000","last_clean":"2023-07-06T02:22:59.643049+0000","last_became_active":"2023-07-06T02:22:57.446786+0000","last_became_peered":"2023-07-06T02:22:57.446786+0000","last_unstale":"2023-07-06T02:22:59.643049+0000","last_undegraded":"2023-07-06T02:22:59.643049+0000","last_fullsized":"2023-07-06T02:22:59.643049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:45.545737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643069+0000","last_change":"2023-07-06T02:22:52.419733+0000","last_active":"2023-07-06T02:22:59.643069+0000","last_peered":"2023-07-06T02:22:59.643069+0000","last_clean":"2023-07-06T02:22:59.643069+0000","last_became_active":"2023-07-06T02:22:52.418972+0000","last_became_peered":"2023-07-06T02:22:52.418972+0000","last_unstale":"2023-07-06T02:22:59.643069+0000","last_undegraded":"2023-07-06T02:22:59.643069+0000","last_fullsized":"2023-07-06T02:22:59.643069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:07:57.245299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622844+0000","last_change":"2023-07-06T02:22:53.416731+0000","last_active":"2023-07-06T02:22:58.622844+0000","last_peered":"2023-07-06T02:22:58.622844+0000","last_clean":"2023-07-06T02:22:58.622844+0000","last_became_active":"2023-07-06T02:22:53.416412+0000","last_became_peered":"2023-07-06T02:22:53.416412+0000","last_unstale":"2023-07-06T02:22:58.622844+0000","last_undegraded":"2023-07-06T02:22:58.622844+0000","last_fullsized":"2023-07-06T02:22:58.622844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:36:20.268819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622716+0000","last_change":"2023-07-06T02:22:57.461025+0000","last_active":"2023-07-06T02:22:58.622716+0000","last_peered":"2023-07-06T02:22:58.622716+0000","last_clean":"2023-07-06T02:22:58.622716+0000","last_became_active":"2023-07-06T02:22:57.460633+0000","last_became_peered":"2023-07-06T02:22:57.460633+0000","last_unstale":"2023-07-06T02:22:58.622716+0000","last_undegraded":"2023-07-06T02:22:58.622716+0000","last_fullsized":"2023-07-06T02:22:58.622716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:26.873513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:52.418771+0000","last_became_peered":"2023-07-06T02:22:52.418771+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:24:21.889565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463056+0000","last_change":"2023-07-06T02:22:53.421840+0000","last_active":"2023-07-06T02:22:58.463056+0000","last_peered":"2023-07-06T02:22:58.463056+0000","last_clean":"2023-07-06T02:22:58.463056+0000","last_became_active":"2023-07-06T02:22:53.421644+0000","last_became_peered":"2023-07-06T02:22:53.421644+0000","last_unstale":"2023-07-06T02:22:58.463056+0000","last_undegraded":"2023-07-06T02:22:58.463056+0000","last_fullsized":"2023-07-06T02:22:58.463056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:42:56.076946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:57.459681+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:57.459298+0000","last_became_peered":"2023-07-06T02:22:57.459298+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:49.871715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642530+0000","last_change":"2023-07-06T02:22:52.425605+0000","last_active":"2023-07-06T02:22:59.642530+0000","last_peered":"2023-07-06T02:22:59.642530+0000","last_clean":"2023-07-06T02:22:59.642530+0000","last_became_active":"2023-07-06T02:22:52.425345+0000","last_became_peered":"2023-07-06T02:22:52.425345+0000","last_unstale":"2023-07-06T02:22:59.642530+0000","last_undegraded":"2023-07-06T02:22:59.642530+0000","last_fullsized":"2023-07-06T02:22:59.642530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622815+0000","last_change":"2023-07-06T02:22:53.422473+0000","last_active":"2023-07-06T02:22:58.622815+0000","last_peered":"2023-07-06T02:22:58.622815+0000","last_clean":"2023-07-06T02:22:58.622815+0000","last_became_active":"2023-07-06T02:22:53.422171+0000","last_became_peered":"2023-07-06T02:22:53.422171+0000","last_unstale":"2023-07-06T02:22:58.622815+0000","last_undegraded":"2023-07-06T02:22:58.622815+0000","last_fullsized":"2023-07-06T02:22:58.622815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:35.957039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462966+0000","last_change":"2023-07-06T02:22:53.420559+0000","last_active":"2023-07-06T02:22:58.462966+0000","last_peered":"2023-07-06T02:22:58.462966+0000","last_clean":"2023-07-06T02:22:58.462966+0000","last_became_active":"2023-07-06T02:22:53.420220+0000","last_became_peered":"2023-07-06T02:22:53.420220+0000","last_unstale":"2023-07-06T02:22:58.462966+0000","last_undegraded":"2023-07-06T02:22:58.462966+0000","last_fullsized":"2023-07-06T02:22:58.462966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:11.617347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462676+0000","last_change":"2023-07-06T02:22:53.424835+0000","last_active":"2023-07-06T02:22:58.462676+0000","last_peered":"2023-07-06T02:22:58.462676+0000","last_clean":"2023-07-06T02:22:58.462676+0000","last_became_active":"2023-07-06T02:22:53.424520+0000","last_became_peered":"2023-07-06T02:22:53.424520+0000","last_unstale":"2023-07-06T02:22:58.462676+0000","last_undegraded":"2023-07-06T02:22:58.462676+0000","last_fullsized":"2023-07-06T02:22:58.462676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:27.623742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623256+0000","last_change":"2023-07-06T02:22:53.420661+0000","last_active":"2023-07-06T02:22:58.623256+0000","last_peered":"2023-07-06T02:22:58.623256+0000","last_clean":"2023-07-06T02:22:58.623256+0000","last_became_active":"2023-07-06T02:22:53.420137+0000","last_became_peered":"2023-07-06T02:22:53.420137+0000","last_unstale":"2023-07-06T02:22:58.623256+0000","last_undegraded":"2023-07-06T02:22:58.623256+0000","last_fullsized":"2023-07-06T02:22:58.623256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:07:51.965833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462455+0000","last_change":"2023-07-06T02:22:53.434477+0000","last_active":"2023-07-06T02:22:58.462455+0000","last_peered":"2023-07-06T02:22:58.462455+0000","last_clean":"2023-07-06T02:22:58.462455+0000","last_became_active":"2023-07-06T02:22:53.434127+0000","last_became_peered":"2023-07-06T02:22:53.434127+0000","last_unstale":"2023-07-06T02:22:58.462455+0000","last_undegraded":"2023-07-06T02:22:58.462455+0000","last_fullsized":"2023-07-06T02:22:58.462455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:53.433904+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:53.433683+0000","last_became_peered":"2023-07-06T02:22:53.433683+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T02:23:12.285 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T02:23:12.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.424+0000 7f0d9602a700 1 Processor -- start 2023-07-06T02:23:12.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.424+0000 7f0d9602a700 1 -- start start 2023-07-06T02:23:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d9602a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d901600b0 0x7f0d900b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d9602a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d900b19d0 0x7f0d900cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d9602a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d900cb3d0 con 0x7f0d90156ab0 2023-07-06T02:23:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d9602a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900b1df0 con 0x7f0d900b19d0 2023-07-06T02:23:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d9602a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d90156e80 con 0x7f0d901600b0 2023-07-06T02:23:12.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d900b19d0 0x7f0d900cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d900b19d0 0x7f0d900cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60724/0 (socket says 172.21.15.89:60724) 2023-07-06T02:23:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 -- 172.21.15.89:0/866554265 learned_addr learned my addr 172.21.15.89:0/866554265 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0d90156ab0 0x7f0d90160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57390/0 (socket says 172.21.15.89:57390) 2023-07-06T02:23:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8effd700 1 --2- 172.21.15.89:0/866554265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d901600b0 0x7f0d900b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 -- 172.21.15.89:0/866554265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d901600b0 msgr2=0x7f0d900b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 --2- 172.21.15.89:0/866554265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d901600b0 0x7f0d900b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 -- 172.21.15.89:0/866554265 >> v1:172.21.15.89:6789/0 conn(0x7f0d90156ab0 legacy=0x7f0d90160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 -- 172.21.15.89:0/866554265 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900ca370 con 0x7f0d900b19d0 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8e7fc700 1 -- 172.21.15.89:0/866554265 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364798200 0 0) 0x7f0d900cb3d0 con 0x7f0d90156ab0 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8ffff700 1 --2- 172.21.15.89:0/866554265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d900b19d0 0x7f0d900cd700 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009e50 tx=0x7f0d8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798e113ed5e5fe24 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8e7fc700 1 -- 172.21.15.89:0/866554265 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800e070 con 0x7f0d900b19d0 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8e7fc700 1 -- 172.21.15.89:0/866554265 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8800b9c0 con 0x7f0d900b19d0 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.425+0000 7f0d8e7fc700 1 -- 172.21.15.89:0/866554265 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800bd70 con 0x7f0d900b19d0 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 -- 172.21.15.89:0/866554265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d900b19d0 msgr2=0x7f0d900cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 --2- 172.21.15.89:0/866554265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d900b19d0 0x7f0d900cd700 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009e50 tx=0x7f0d8800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 -- 172.21.15.89:0/866554265 shutdown_connections 2023-07-06T02:23:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 --2- 172.21.15.89:0/866554265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d900b19d0 0x7f0d900cd700 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 --2- 172.21.15.89:0/866554265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d901600b0 0x7f0d900b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 -- 172.21.15.89:0/866554265 >> 172.21.15.89:0/866554265 conn(0x7f0d900cc490 msgr2=0x7f0d900cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 -- 172.21.15.89:0/866554265 shutdown_connections 2023-07-06T02:23:12.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.426+0000 7f0d9602a700 1 -- 172.21.15.89:0/866554265 wait complete. 2023-07-06T02:23:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.427+0000 7f0d9602a700 1 Processor -- start 2023-07-06T02:23:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.427+0000 7f0d9602a700 1 -- start start 2023-07-06T02:23:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d9602a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d900b19d0 0x7f0d901e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d9602a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d90156ab0 0x7f0d901f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d9602a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d901600b0 0x7f0d901f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d9602a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d901e6730 con 0x7f0d90156ab0 2023-07-06T02:23:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d9602a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900c4650 con 0x7f0d901600b0 2023-07-06T02:23:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d9602a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900c4930 con 0x7f0d900b19d0 2023-07-06T02:23:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d90156ab0 0x7f0d901f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d90156ab0 0x7f0d901f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60728/0 (socket says 172.21.15.89:60728) 2023-07-06T02:23:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 -- 172.21.15.89:0/725826376 learned_addr learned my addr 172.21.15.89:0/725826376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8ffff700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d901600b0 0x7f0d901f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8f7fe700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d900b19d0 0x7f0d901e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 -- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d900b19d0 msgr2=0x7f0d901e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d900b19d0 0x7f0d901e61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 -- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d901600b0 msgr2=0x7f0d901f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d901600b0 0x7f0d901f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.428+0000 7f0d8effd700 1 -- 172.21.15.89:0/725826376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009b30 con 0x7f0d90156ab0 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.429+0000 7f0d8effd700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d90156ab0 0x7f0d901f0d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400d800 tx=0x7f0d8400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.429+0000 7f0d77fff700 1 -- 172.21.15.89:0/725826376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d840125f0 con 0x7f0d90156ab0 2023-07-06T02:23:12.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.429+0000 7f0d77fff700 1 -- 172.21.15.89:0/725826376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84012750 con 0x7f0d90156ab0 2023-07-06T02:23:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.429+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d900c4c10 con 0x7f0d90156ab0 2023-07-06T02:23:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.429+0000 7f0d77fff700 1 -- 172.21.15.89:0/725826376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84012b00 con 0x7f0d90156ab0 2023-07-06T02:23:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.429+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d900c51d0 con 0x7f0d90156ab0 2023-07-06T02:23:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.430+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d900c5390 con 0x7f0d90156ab0 2023-07-06T02:23:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.430+0000 7f0d77fff700 1 -- 172.21.15.89:0/725826376 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d84012c60 con 0x7f0d90156ab0 2023-07-06T02:23:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.430+0000 7f0d77fff700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0d7c06ace0 0x7f0d7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.431+0000 7f0d8f7fe700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0d7c06ace0 0x7f0d7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.431+0000 7f0d77fff700 1 -- 172.21.15.89:0/725826376 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0d84089b60 con 0x7f0d90156ab0 2023-07-06T02:23:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.431+0000 7f0d8f7fe700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0d7c06ace0 0x7f0d7c06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000f60 tx=0x7f0d80007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.435+0000 7f0d77fff700 1 -- 172.21.15.89:0/725826376 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0d84054920 con 0x7f0d90156ab0 2023-07-06T02:23:12.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.621+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 --> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d90160d80 con 0x7f0d7c06ace0 2023-07-06T02:23:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.630+0000 7f0d77fff700 1 -- 172.21.15.89:0/725826376 <== mgr.4100 v2:172.21.15.89:6832/3591115711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367553 (secure 0 0 0) 0x7f0d800012b0 con 0x7f0d7c06ace0 2023-07-06T02:23:12.637 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:12.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0d7c06ace0 msgr2=0x7f0d7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0d7c06ace0 0x7f0d7c06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000f60 tx=0x7f0d80007be0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d90156ab0 msgr2=0x7f0d901f0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d90156ab0 0x7f0d901f0d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400d800 tx=0x7f0d8400dbc0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 shutdown_connections 2023-07-06T02:23:12.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d901600b0 0x7f0d901f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d900b19d0 0x7f0d901e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0d7c06ace0 0x7f0d7c06d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 --2- 172.21.15.89:0/725826376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d90156ab0 0x7f0d901f0d10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 >> 172.21.15.89:0/725826376 conn(0x7f0d900cc490 msgr2=0x7f0d9015f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:12.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 shutdown_connections 2023-07-06T02:23:12.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.634+0000 7f0d9602a700 1 -- 172.21.15.89:0/725826376 wait complete. 2023-07-06T02:23:12.641 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-07-06T02:23:12.649 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-07-06T02:23:11.981525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218820,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755900,"statfs":{"total":773094113280,"available":772870041600,"internally_reserved":0,"allocated":3280896,"data_stored":2026399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002677"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462978+0000","last_change":"2023-07-06T02:22:53.433054+0000","last_active":"2023-07-06T02:22:58.462978+0000","last_peered":"2023-07-06T02:22:58.462978+0000","last_clean":"2023-07-06T02:22:58.462978+0000","last_became_active":"2023-07-06T02:22:53.432749+0000","last_became_peered":"2023-07-06T02:22:53.432749+0000","last_unstale":"2023-07-06T02:22:58.462978+0000","last_undegraded":"2023-07-06T02:22:58.462978+0000","last_fullsized":"2023-07-06T02:22:58.462978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:42.597214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643097+0000","last_change":"2023-07-06T02:22:53.425731+0000","last_active":"2023-07-06T02:22:59.643097+0000","last_peered":"2023-07-06T02:22:59.643097+0000","last_clean":"2023-07-06T02:22:59.643097+0000","last_became_active":"2023-07-06T02:22:53.425403+0000","last_became_peered":"2023-07-06T02:22:53.425403+0000","last_unstale":"2023-07-06T02:22:59.643097+0000","last_undegraded":"2023-07-06T02:22:59.643097+0000","last_fullsized":"2023-07-06T02:22:59.643097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:00.409252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623835+0000","last_change":"2023-07-06T02:22:53.413131+0000","last_active":"2023-07-06T02:22:58.623835+0000","last_peered":"2023-07-06T02:22:58.623835+0000","last_clean":"2023-07-06T02:22:58.623835+0000","last_became_active":"2023-07-06T02:22:53.412839+0000","last_became_peered":"2023-07-06T02:22:53.412839+0000","last_unstale":"2023-07-06T02:22:58.623835+0000","last_undegraded":"2023-07-06T02:22:58.623835+0000","last_fullsized":"2023-07-06T02:22:58.623835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:51.080641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463166+0000","last_change":"2023-07-06T02:22:53.420056+0000","last_active":"2023-07-06T02:22:58.463166+0000","last_peered":"2023-07-06T02:22:58.463166+0000","last_clean":"2023-07-06T02:22:58.463166+0000","last_became_active":"2023-07-06T02:22:53.419779+0000","last_became_peered":"2023-07-06T02:22:53.419779+0000","last_unstale":"2023-07-06T02:22:58.463166+0000","last_undegraded":"2023-07-06T02:22:58.463166+0000","last_fullsized":"2023-07-06T02:22:58.463166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:37.102968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642705+0000","last_change":"2023-07-06T02:22:53.420867+0000","last_active":"2023-07-06T02:22:59.642705+0000","last_peered":"2023-07-06T02:22:59.642705+0000","last_clean":"2023-07-06T02:22:59.642705+0000","last_became_active":"2023-07-06T02:22:53.419716+0000","last_became_peered":"2023-07-06T02:22:53.419716+0000","last_unstale":"2023-07-06T02:22:59.642705+0000","last_undegraded":"2023-07-06T02:22:59.642705+0000","last_fullsized":"2023-07-06T02:22:59.642705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:33.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623263+0000","last_change":"2023-07-06T02:22:53.412521+0000","last_active":"2023-07-06T02:22:58.623263+0000","last_peered":"2023-07-06T02:22:58.623263+0000","last_clean":"2023-07-06T02:22:58.623263+0000","last_became_active":"2023-07-06T02:22:53.412230+0000","last_became_peered":"2023-07-06T02:22:53.412230+0000","last_unstale":"2023-07-06T02:22:58.623263+0000","last_undegraded":"2023-07-06T02:22:58.623263+0000","last_fullsized":"2023-07-06T02:22:58.623263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:44.353524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545634+0000","last_change":"2023-07-06T02:22:53.427954+0000","last_active":"2023-07-06T02:22:58.545634+0000","last_peered":"2023-07-06T02:22:58.545634+0000","last_clean":"2023-07-06T02:22:58.545634+0000","last_became_active":"2023-07-06T02:22:53.426315+0000","last_became_peered":"2023-07-06T02:22:53.426315+0000","last_unstale":"2023-07-06T02:22:58.545634+0000","last_undegraded":"2023-07-06T02:22:58.545634+0000","last_fullsized":"2023-07-06T02:22:58.545634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642615+0000","last_change":"2023-07-06T02:22:53.425882+0000","last_active":"2023-07-06T02:22:59.642615+0000","last_peered":"2023-07-06T02:22:59.642615+0000","last_clean":"2023-07-06T02:22:59.642615+0000","last_became_active":"2023-07-06T02:22:53.425583+0000","last_became_peered":"2023-07-06T02:22:53.425583+0000","last_unstale":"2023-07-06T02:22:59.642615+0000","last_undegraded":"2023-07-06T02:22:59.642615+0000","last_fullsized":"2023-07-06T02:22:59.642615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:51.387481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642268+0000","last_change":"2023-07-06T02:22:53.427106+0000","last_active":"2023-07-06T02:22:59.642268+0000","last_peered":"2023-07-06T02:22:59.642268+0000","last_clean":"2023-07-06T02:22:59.642268+0000","last_became_active":"2023-07-06T02:22:53.426852+0000","last_became_peered":"2023-07-06T02:22:53.426852+0000","last_unstale":"2023-07-06T02:22:59.642268+0000","last_undegraded":"2023-07-06T02:22:59.642268+0000","last_fullsized":"2023-07-06T02:22:59.642268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643507+0000","last_change":"2023-07-06T02:22:53.421592+0000","last_active":"2023-07-06T02:22:59.643507+0000","last_peered":"2023-07-06T02:22:59.643507+0000","last_clean":"2023-07-06T02:22:59.643507+0000","last_became_active":"2023-07-06T02:22:53.421245+0000","last_became_peered":"2023-07-06T02:22:53.421245+0000","last_unstale":"2023-07-06T02:22:59.643507+0000","last_undegraded":"2023-07-06T02:22:59.643507+0000","last_fullsized":"2023-07-06T02:22:59.643507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623746+0000","last_change":"2023-07-06T02:22:53.424792+0000","last_active":"2023-07-06T02:22:58.623746+0000","last_peered":"2023-07-06T02:22:58.623746+0000","last_clean":"2023-07-06T02:22:58.623746+0000","last_became_active":"2023-07-06T02:22:53.424498+0000","last_became_peered":"2023-07-06T02:22:53.424498+0000","last_unstale":"2023-07-06T02:22:58.623746+0000","last_undegraded":"2023-07-06T02:22:58.623746+0000","last_fullsized":"2023-07-06T02:22:58.623746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:55:40.483266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462649+0000","last_change":"2023-07-06T02:22:53.424739+0000","last_active":"2023-07-06T02:22:58.462649+0000","last_peered":"2023-07-06T02:22:58.462649+0000","last_clean":"2023-07-06T02:22:58.462649+0000","last_became_active":"2023-07-06T02:22:53.424382+0000","last_became_peered":"2023-07-06T02:22:53.424382+0000","last_unstale":"2023-07-06T02:22:58.462649+0000","last_undegraded":"2023-07-06T02:22:58.462649+0000","last_fullsized":"2023-07-06T02:22:58.462649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:08:50.637616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462913+0000","last_change":"2023-07-06T02:22:53.435357+0000","last_active":"2023-07-06T02:22:58.462913+0000","last_peered":"2023-07-06T02:22:58.462913+0000","last_clean":"2023-07-06T02:22:58.462913+0000","last_became_active":"2023-07-06T02:22:53.435068+0000","last_became_peered":"2023-07-06T02:22:53.435068+0000","last_unstale":"2023-07-06T02:22:58.462913+0000","last_undegraded":"2023-07-06T02:22:58.462913+0000","last_fullsized":"2023-07-06T02:22:58.462913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:05.764438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642840+0000","last_change":"2023-07-06T02:22:53.419307+0000","last_active":"2023-07-06T02:22:59.642840+0000","last_peered":"2023-07-06T02:22:59.642840+0000","last_clean":"2023-07-06T02:22:59.642840+0000","last_became_active":"2023-07-06T02:22:53.419010+0000","last_became_peered":"2023-07-06T02:22:53.419010+0000","last_unstale":"2023-07-06T02:22:59.642840+0000","last_undegraded":"2023-07-06T02:22:59.642840+0000","last_fullsized":"2023-07-06T02:22:59.642840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:52.740513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545431+0000","last_change":"2023-07-06T02:22:53.427932+0000","last_active":"2023-07-06T02:22:58.545431+0000","last_peered":"2023-07-06T02:22:58.545431+0000","last_clean":"2023-07-06T02:22:58.545431+0000","last_became_active":"2023-07-06T02:22:53.426280+0000","last_became_peered":"2023-07-06T02:22:53.426280+0000","last_unstale":"2023-07-06T02:22:58.545431+0000","last_undegraded":"2023-07-06T02:22:58.545431+0000","last_fullsized":"2023-07-06T02:22:58.545431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:05.507588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622966+0000","last_change":"2023-07-06T02:22:53.412012+0000","last_active":"2023-07-06T02:22:58.622966+0000","last_peered":"2023-07-06T02:22:58.622966+0000","last_clean":"2023-07-06T02:22:58.622966+0000","last_became_active":"2023-07-06T02:22:53.411675+0000","last_became_peered":"2023-07-06T02:22:53.411675+0000","last_unstale":"2023-07-06T02:22:58.622966+0000","last_undegraded":"2023-07-06T02:22:58.622966+0000","last_fullsized":"2023-07-06T02:22:58.622966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:13:54.565351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:53.432526+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:53.432240+0000","last_became_peered":"2023-07-06T02:22:53.432240+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623666+0000","last_change":"2023-07-06T02:22:53.419854+0000","last_active":"2023-07-06T02:22:58.623666+0000","last_peered":"2023-07-06T02:22:58.623666+0000","last_clean":"2023-07-06T02:22:58.623666+0000","last_became_active":"2023-07-06T02:22:53.419569+0000","last_became_peered":"2023-07-06T02:22:53.419569+0000","last_unstale":"2023-07-06T02:22:58.623666+0000","last_undegraded":"2023-07-06T02:22:58.623666+0000","last_fullsized":"2023-07-06T02:22:58.623666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:44:47.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463157+0000","last_change":"2023-07-06T02:22:53.422202+0000","last_active":"2023-07-06T02:22:58.463157+0000","last_peered":"2023-07-06T02:22:58.463157+0000","last_clean":"2023-07-06T02:22:58.463157+0000","last_became_active":"2023-07-06T02:22:53.421981+0000","last_became_peered":"2023-07-06T02:22:53.421981+0000","last_unstale":"2023-07-06T02:22:58.463157+0000","last_undegraded":"2023-07-06T02:22:58.463157+0000","last_fullsized":"2023-07-06T02:22:58.463157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:55.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623252+0000","last_change":"2023-07-06T02:22:53.421294+0000","last_active":"2023-07-06T02:22:58.623252+0000","last_peered":"2023-07-06T02:22:58.623252+0000","last_clean":"2023-07-06T02:22:58.623252+0000","last_became_active":"2023-07-06T02:22:53.420987+0000","last_became_peered":"2023-07-06T02:22:53.420987+0000","last_unstale":"2023-07-06T02:22:58.623252+0000","last_undegraded":"2023-07-06T02:22:58.623252+0000","last_fullsized":"2023-07-06T02:22:58.623252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:07:23.523735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545995+0000","last_change":"2023-07-06T02:22:53.423456+0000","last_active":"2023-07-06T02:22:58.545995+0000","last_peered":"2023-07-06T02:22:58.545995+0000","last_clean":"2023-07-06T02:22:58.545995+0000","last_became_active":"2023-07-06T02:22:53.423131+0000","last_became_peered":"2023-07-06T02:22:53.423131+0000","last_unstale":"2023-07-06T02:22:58.545995+0000","last_undegraded":"2023-07-06T02:22:58.545995+0000","last_fullsized":"2023-07-06T02:22:58.545995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641976+0000","last_change":"2023-07-06T02:22:53.424047+0000","last_active":"2023-07-06T02:22:59.641976+0000","last_peered":"2023-07-06T02:22:59.641976+0000","last_clean":"2023-07-06T02:22:59.641976+0000","last_became_active":"2023-07-06T02:22:53.423692+0000","last_became_peered":"2023-07-06T02:22:53.423692+0000","last_unstale":"2023-07-06T02:22:59.641976+0000","last_undegraded":"2023-07-06T02:22:59.641976+0000","last_fullsized":"2023-07-06T02:22:59.641976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.432531+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.432245+0000","last_became_peered":"2023-07-06T02:22:53.432245+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:09:56.578120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545462+0000","last_change":"2023-07-06T02:22:53.426016+0000","last_active":"2023-07-06T02:22:58.545462+0000","last_peered":"2023-07-06T02:22:58.545462+0000","last_clean":"2023-07-06T02:22:58.545462+0000","last_became_active":"2023-07-06T02:22:53.417030+0000","last_became_peered":"2023-07-06T02:22:53.417030+0000","last_unstale":"2023-07-06T02:22:58.545462+0000","last_undegraded":"2023-07-06T02:22:58.545462+0000","last_fullsized":"2023-07-06T02:22:58.545462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497209+0000","last_change":"2023-07-06T02:22:53.422907+0000","last_active":"2023-07-06T02:22:58.497209+0000","last_peered":"2023-07-06T02:22:58.497209+0000","last_clean":"2023-07-06T02:22:58.497209+0000","last_became_active":"2023-07-06T02:22:53.422617+0000","last_became_peered":"2023-07-06T02:22:53.422617+0000","last_unstale":"2023-07-06T02:22:58.497209+0000","last_undegraded":"2023-07-06T02:22:58.497209+0000","last_fullsized":"2023-07-06T02:22:58.497209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:41.517576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497094+0000","last_change":"2023-07-06T02:22:53.422413+0000","last_active":"2023-07-06T02:22:58.497094+0000","last_peered":"2023-07-06T02:22:58.497094+0000","last_clean":"2023-07-06T02:22:58.497094+0000","last_became_active":"2023-07-06T02:22:53.422025+0000","last_became_peered":"2023-07-06T02:22:53.422025+0000","last_unstale":"2023-07-06T02:22:58.497094+0000","last_undegraded":"2023-07-06T02:22:58.497094+0000","last_fullsized":"2023-07-06T02:22:58.497094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:23.425050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643229+0000","last_change":"2023-07-06T02:22:53.434688+0000","last_active":"2023-07-06T02:22:59.643229+0000","last_peered":"2023-07-06T02:22:59.643229+0000","last_clean":"2023-07-06T02:22:59.643229+0000","last_became_active":"2023-07-06T02:22:53.434423+0000","last_became_peered":"2023-07-06T02:22:53.434423+0000","last_unstale":"2023-07-06T02:22:59.643229+0000","last_undegraded":"2023-07-06T02:22:59.643229+0000","last_fullsized":"2023-07-06T02:22:59.643229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:11.004232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623583+0000","last_change":"2023-07-06T02:22:57.463283+0000","last_active":"2023-07-06T02:22:58.623583+0000","last_peered":"2023-07-06T02:22:58.623583+0000","last_clean":"2023-07-06T02:22:58.623583+0000","last_became_active":"2023-07-06T02:22:57.462913+0000","last_became_peered":"2023-07-06T02:22:57.462913+0000","last_unstale":"2023-07-06T02:22:58.623583+0000","last_undegraded":"2023-07-06T02:22:58.623583+0000","last_fullsized":"2023-07-06T02:22:58.623583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:29.980455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623061+0000","last_change":"2023-07-06T02:22:52.423689+0000","last_active":"2023-07-06T02:22:58.623061+0000","last_peered":"2023-07-06T02:22:58.623061+0000","last_clean":"2023-07-06T02:22:58.623061+0000","last_became_active":"2023-07-06T02:22:52.423464+0000","last_became_peered":"2023-07-06T02:22:52.423464+0000","last_unstale":"2023-07-06T02:22:58.623061+0000","last_undegraded":"2023-07-06T02:22:58.623061+0000","last_fullsized":"2023-07-06T02:22:58.623061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:34.698538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463653+0000","last_change":"2023-07-06T02:22:53.426231+0000","last_active":"2023-07-06T02:22:58.463653+0000","last_peered":"2023-07-06T02:22:58.463653+0000","last_clean":"2023-07-06T02:22:58.463653+0000","last_became_active":"2023-07-06T02:22:53.425923+0000","last_became_peered":"2023-07-06T02:22:53.425923+0000","last_unstale":"2023-07-06T02:22:58.463653+0000","last_undegraded":"2023-07-06T02:22:58.463653+0000","last_fullsized":"2023-07-06T02:22:58.463653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:22.845352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463216+0000","last_change":"2023-07-06T02:22:57.461661+0000","last_active":"2023-07-06T02:22:58.463216+0000","last_peered":"2023-07-06T02:22:58.463216+0000","last_clean":"2023-07-06T02:22:58.463216+0000","last_became_active":"2023-07-06T02:22:57.461262+0000","last_became_peered":"2023-07-06T02:22:57.461262+0000","last_unstale":"2023-07-06T02:22:58.463216+0000","last_undegraded":"2023-07-06T02:22:58.463216+0000","last_fullsized":"2023-07-06T02:22:58.463216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:05.846802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496610+0000","last_change":"2023-07-06T02:22:52.424261+0000","last_active":"2023-07-06T02:22:58.496610+0000","last_peered":"2023-07-06T02:22:58.496610+0000","last_clean":"2023-07-06T02:22:58.496610+0000","last_became_active":"2023-07-06T02:22:52.423984+0000","last_became_peered":"2023-07-06T02:22:52.423984+0000","last_unstale":"2023-07-06T02:22:58.496610+0000","last_undegraded":"2023-07-06T02:22:58.496610+0000","last_fullsized":"2023-07-06T02:22:58.496610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:56.520485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642313+0000","last_change":"2023-07-06T02:22:53.417163+0000","last_active":"2023-07-06T02:22:59.642313+0000","last_peered":"2023-07-06T02:22:59.642313+0000","last_clean":"2023-07-06T02:22:59.642313+0000","last_became_active":"2023-07-06T02:22:53.416875+0000","last_became_peered":"2023-07-06T02:22:53.416875+0000","last_unstale":"2023-07-06T02:22:59.642313+0000","last_undegraded":"2023-07-06T02:22:59.642313+0000","last_fullsized":"2023-07-06T02:22:59.642313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642441+0000","last_change":"2023-07-06T02:22:57.455750+0000","last_active":"2023-07-06T02:22:59.642441+0000","last_peered":"2023-07-06T02:22:59.642441+0000","last_clean":"2023-07-06T02:22:59.642441+0000","last_became_active":"2023-07-06T02:22:57.455331+0000","last_became_peered":"2023-07-06T02:22:57.455331+0000","last_unstale":"2023-07-06T02:22:59.642441+0000","last_undegraded":"2023-07-06T02:22:59.642441+0000","last_fullsized":"2023-07-06T02:22:59.642441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462903+0000","last_change":"2023-07-06T02:22:52.416577+0000","last_active":"2023-07-06T02:22:58.462903+0000","last_peered":"2023-07-06T02:22:58.462903+0000","last_clean":"2023-07-06T02:22:58.462903+0000","last_became_active":"2023-07-06T02:22:52.416144+0000","last_became_peered":"2023-07-06T02:22:52.416144+0000","last_unstale":"2023-07-06T02:22:58.462903+0000","last_undegraded":"2023-07-06T02:22:58.462903+0000","last_fullsized":"2023-07-06T02:22:58.462903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:03:22.897984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463486+0000","last_change":"2023-07-06T02:22:53.429089+0000","last_active":"2023-07-06T02:22:58.463486+0000","last_peered":"2023-07-06T02:22:58.463486+0000","last_clean":"2023-07-06T02:22:58.463486+0000","last_became_active":"2023-07-06T02:22:53.428809+0000","last_became_peered":"2023-07-06T02:22:53.428809+0000","last_unstale":"2023-07-06T02:22:58.463486+0000","last_undegraded":"2023-07-06T02:22:58.463486+0000","last_fullsized":"2023-07-06T02:22:58.463486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:30:16.217797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462711+0000","last_change":"2023-07-06T02:22:57.455092+0000","last_active":"2023-07-06T02:22:58.462711+0000","last_peered":"2023-07-06T02:22:58.462711+0000","last_clean":"2023-07-06T02:22:58.462711+0000","last_became_active":"2023-07-06T02:22:57.454707+0000","last_became_peered":"2023-07-06T02:22:57.454707+0000","last_unstale":"2023-07-06T02:22:58.462711+0000","last_undegraded":"2023-07-06T02:22:58.462711+0000","last_fullsized":"2023-07-06T02:22:58.462711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:09:16.226734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:52.428104+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:52.427863+0000","last_became_peered":"2023-07-06T02:22:52.427863+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:52.064862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.424797+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.424474+0000","last_became_peered":"2023-07-06T02:22:53.424474+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:31:13.843201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545356+0000","last_change":"2023-07-06T02:22:57.457309+0000","last_active":"2023-07-06T02:22:58.545356+0000","last_peered":"2023-07-06T02:22:58.545356+0000","last_clean":"2023-07-06T02:22:58.545356+0000","last_became_active":"2023-07-06T02:22:57.457008+0000","last_became_peered":"2023-07-06T02:22:57.457008+0000","last_unstale":"2023-07-06T02:22:58.545356+0000","last_undegraded":"2023-07-06T02:22:58.545356+0000","last_fullsized":"2023-07-06T02:22:58.545356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:04.437413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643326+0000","last_change":"2023-07-06T02:22:52.423048+0000","last_active":"2023-07-06T02:22:59.643326+0000","last_peered":"2023-07-06T02:22:59.643326+0000","last_clean":"2023-07-06T02:22:59.643326+0000","last_became_active":"2023-07-06T02:22:52.422801+0000","last_became_peered":"2023-07-06T02:22:52.422801+0000","last_unstale":"2023-07-06T02:22:59.643326+0000","last_undegraded":"2023-07-06T02:22:59.643326+0000","last_fullsized":"2023-07-06T02:22:59.643326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:29.807668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462478+0000","last_change":"2023-07-06T02:22:53.434004+0000","last_active":"2023-07-06T02:22:58.462478+0000","last_peered":"2023-07-06T02:22:58.462478+0000","last_clean":"2023-07-06T02:22:58.462478+0000","last_became_active":"2023-07-06T02:22:53.433739+0000","last_became_peered":"2023-07-06T02:22:53.433739+0000","last_unstale":"2023-07-06T02:22:58.462478+0000","last_undegraded":"2023-07-06T02:22:58.462478+0000","last_fullsized":"2023-07-06T02:22:58.462478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463415+0000","last_change":"2023-07-06T02:22:57.452160+0000","last_active":"2023-07-06T02:22:58.463415+0000","last_peered":"2023-07-06T02:22:58.463415+0000","last_clean":"2023-07-06T02:22:58.463415+0000","last_became_active":"2023-07-06T02:22:57.451507+0000","last_became_peered":"2023-07-06T02:22:57.451507+0000","last_unstale":"2023-07-06T02:22:58.463415+0000","last_undegraded":"2023-07-06T02:22:58.463415+0000","last_fullsized":"2023-07-06T02:22:58.463415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:10:18.060753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546029+0000","last_change":"2023-07-06T02:22:52.419096+0000","last_active":"2023-07-06T02:22:58.546029+0000","last_peered":"2023-07-06T02:22:58.546029+0000","last_clean":"2023-07-06T02:22:58.546029+0000","last_became_active":"2023-07-06T02:22:52.418844+0000","last_became_peered":"2023-07-06T02:22:52.418844+0000","last_unstale":"2023-07-06T02:22:58.546029+0000","last_undegraded":"2023-07-06T02:22:58.546029+0000","last_fullsized":"2023-07-06T02:22:58.546029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:14.974840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496797+0000","last_change":"2023-07-06T02:22:53.425547+0000","last_active":"2023-07-06T02:22:58.496797+0000","last_peered":"2023-07-06T02:22:58.496797+0000","last_clean":"2023-07-06T02:22:58.496797+0000","last_became_active":"2023-07-06T02:22:53.425239+0000","last_became_peered":"2023-07-06T02:22:53.425239+0000","last_unstale":"2023-07-06T02:22:58.496797+0000","last_undegraded":"2023-07-06T02:22:58.496797+0000","last_fullsized":"2023-07-06T02:22:58.496797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:32.244431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496944+0000","last_change":"2023-07-06T02:22:57.453836+0000","last_active":"2023-07-06T02:22:58.496944+0000","last_peered":"2023-07-06T02:22:58.496944+0000","last_clean":"2023-07-06T02:22:58.496944+0000","last_became_active":"2023-07-06T02:22:57.451637+0000","last_became_peered":"2023-07-06T02:22:57.451637+0000","last_unstale":"2023-07-06T02:22:58.496944+0000","last_undegraded":"2023-07-06T02:22:58.496944+0000","last_fullsized":"2023-07-06T02:22:58.496944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:45.805424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:23:03.321857+0000","last_change":"2023-07-06T02:22:52.421021+0000","last_active":"2023-07-06T02:23:03.321857+0000","last_peered":"2023-07-06T02:23:03.321857+0000","last_clean":"2023-07-06T02:23:03.321857+0000","last_became_active":"2023-07-06T02:22:52.420768+0000","last_became_peered":"2023-07-06T02:22:52.420768+0000","last_unstale":"2023-07-06T02:23:03.321857+0000","last_undegraded":"2023-07-06T02:23:03.321857+0000","last_fullsized":"2023-07-06T02:23:03.321857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:10:06.879321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623414+0000","last_change":"2023-07-06T02:22:53.420729+0000","last_active":"2023-07-06T02:22:58.623414+0000","last_peered":"2023-07-06T02:22:58.623414+0000","last_clean":"2023-07-06T02:22:58.623414+0000","last_became_active":"2023-07-06T02:22:53.418685+0000","last_became_peered":"2023-07-06T02:22:53.418685+0000","last_unstale":"2023-07-06T02:22:58.623414+0000","last_undegraded":"2023-07-06T02:22:58.623414+0000","last_fullsized":"2023-07-06T02:22:58.623414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:47.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643458+0000","last_change":"2023-07-06T02:22:57.455310+0000","last_active":"2023-07-06T02:22:59.643458+0000","last_peered":"2023-07-06T02:22:59.643458+0000","last_clean":"2023-07-06T02:22:59.643458+0000","last_became_active":"2023-07-06T02:22:57.454518+0000","last_became_peered":"2023-07-06T02:22:57.454518+0000","last_unstale":"2023-07-06T02:22:59.643458+0000","last_undegraded":"2023-07-06T02:22:59.643458+0000","last_fullsized":"2023-07-06T02:22:59.643458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:57.100343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545849+0000","last_change":"2023-07-06T02:22:52.421399+0000","last_active":"2023-07-06T02:22:58.545849+0000","last_peered":"2023-07-06T02:22:58.545849+0000","last_clean":"2023-07-06T02:22:58.545849+0000","last_became_active":"2023-07-06T02:22:52.420062+0000","last_became_peered":"2023-07-06T02:22:52.420062+0000","last_unstale":"2023-07-06T02:22:58.545849+0000","last_undegraded":"2023-07-06T02:22:58.545849+0000","last_fullsized":"2023-07-06T02:22:58.545849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:53.425567+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:53.425272+0000","last_became_peered":"2023-07-06T02:22:53.425272+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:45.959463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463853+0000","last_change":"2023-07-06T02:22:57.452727+0000","last_active":"2023-07-06T02:22:58.463853+0000","last_peered":"2023-07-06T02:22:58.463853+0000","last_clean":"2023-07-06T02:22:58.463853+0000","last_became_active":"2023-07-06T02:22:57.452043+0000","last_became_peered":"2023-07-06T02:22:57.452043+0000","last_unstale":"2023-07-06T02:22:58.463853+0000","last_undegraded":"2023-07-06T02:22:58.463853+0000","last_fullsized":"2023-07-06T02:22:58.463853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:26.074085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642190+0000","last_change":"2023-07-06T02:22:52.420802+0000","last_active":"2023-07-06T02:22:59.642190+0000","last_peered":"2023-07-06T02:22:59.642190+0000","last_clean":"2023-07-06T02:22:59.642190+0000","last_became_active":"2023-07-06T02:22:52.420494+0000","last_became_peered":"2023-07-06T02:22:52.420494+0000","last_unstale":"2023-07-06T02:22:59.642190+0000","last_undegraded":"2023-07-06T02:22:59.642190+0000","last_fullsized":"2023-07-06T02:22:59.642190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:04.500268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643228+0000","last_change":"2023-07-06T02:22:53.432233+0000","last_active":"2023-07-06T02:22:59.643228+0000","last_peered":"2023-07-06T02:22:59.643228+0000","last_clean":"2023-07-06T02:22:59.643228+0000","last_became_active":"2023-07-06T02:22:53.431893+0000","last_became_peered":"2023-07-06T02:22:53.431893+0000","last_unstale":"2023-07-06T02:22:59.643228+0000","last_undegraded":"2023-07-06T02:22:59.643228+0000","last_fullsized":"2023-07-06T02:22:59.643228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463256+0000","last_change":"2023-07-06T02:22:57.455213+0000","last_active":"2023-07-06T02:22:58.463256+0000","last_peered":"2023-07-06T02:22:58.463256+0000","last_clean":"2023-07-06T02:22:58.463256+0000","last_became_active":"2023-07-06T02:22:57.454800+0000","last_became_peered":"2023-07-06T02:22:57.454800+0000","last_unstale":"2023-07-06T02:22:58.463256+0000","last_undegraded":"2023-07-06T02:22:58.463256+0000","last_fullsized":"2023-07-06T02:22:58.463256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:00.449720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545483+0000","last_change":"2023-07-06T02:22:52.421475+0000","last_active":"2023-07-06T02:22:58.545483+0000","last_peered":"2023-07-06T02:22:58.545483+0000","last_clean":"2023-07-06T02:22:58.545483+0000","last_became_active":"2023-07-06T02:22:52.421267+0000","last_became_peered":"2023-07-06T02:22:52.421267+0000","last_unstale":"2023-07-06T02:22:58.545483+0000","last_undegraded":"2023-07-06T02:22:58.545483+0000","last_fullsized":"2023-07-06T02:22:58.545483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:09.996912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642587+0000","last_change":"2023-07-06T02:22:53.420060+0000","last_active":"2023-07-06T02:22:59.642587+0000","last_peered":"2023-07-06T02:22:59.642587+0000","last_clean":"2023-07-06T02:22:59.642587+0000","last_became_active":"2023-07-06T02:22:53.419347+0000","last_became_peered":"2023-07-06T02:22:53.419347+0000","last_unstale":"2023-07-06T02:22:59.642587+0000","last_undegraded":"2023-07-06T02:22:59.642587+0000","last_fullsized":"2023-07-06T02:22:59.642587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643160+0000","last_change":"2023-07-06T02:22:57.453231+0000","last_active":"2023-07-06T02:22:59.643160+0000","last_peered":"2023-07-06T02:22:59.643160+0000","last_clean":"2023-07-06T02:22:59.643160+0000","last_became_active":"2023-07-06T02:22:57.452776+0000","last_became_peered":"2023-07-06T02:22:57.452776+0000","last_unstale":"2023-07-06T02:22:59.643160+0000","last_undegraded":"2023-07-06T02:22:59.643160+0000","last_fullsized":"2023-07-06T02:22:59.643160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:09.535749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462794+0000","last_change":"2023-07-06T02:22:52.417281+0000","last_active":"2023-07-06T02:22:58.462794+0000","last_peered":"2023-07-06T02:22:58.462794+0000","last_clean":"2023-07-06T02:22:58.462794+0000","last_became_active":"2023-07-06T02:22:52.417011+0000","last_became_peered":"2023-07-06T02:22:52.417011+0000","last_unstale":"2023-07-06T02:22:58.462794+0000","last_undegraded":"2023-07-06T02:22:58.462794+0000","last_fullsized":"2023-07-06T02:22:58.462794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:56.885245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463676+0000","last_change":"2023-07-06T02:22:53.433973+0000","last_active":"2023-07-06T02:22:58.463676+0000","last_peered":"2023-07-06T02:22:58.463676+0000","last_clean":"2023-07-06T02:22:58.463676+0000","last_became_active":"2023-07-06T02:22:53.433561+0000","last_became_peered":"2023-07-06T02:22:53.433561+0000","last_unstale":"2023-07-06T02:22:58.463676+0000","last_undegraded":"2023-07-06T02:22:58.463676+0000","last_fullsized":"2023-07-06T02:22:58.463676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:36.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545710+0000","last_change":"2023-07-06T02:22:57.455016+0000","last_active":"2023-07-06T02:22:58.545710+0000","last_peered":"2023-07-06T02:22:58.545710+0000","last_clean":"2023-07-06T02:22:58.545710+0000","last_became_active":"2023-07-06T02:22:57.454627+0000","last_became_peered":"2023-07-06T02:22:57.454627+0000","last_unstale":"2023-07-06T02:22:58.545710+0000","last_undegraded":"2023-07-06T02:22:58.545710+0000","last_fullsized":"2023-07-06T02:22:58.545710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546232+0000","last_change":"2023-07-06T02:22:52.420410+0000","last_active":"2023-07-06T02:22:58.546232+0000","last_peered":"2023-07-06T02:22:58.546232+0000","last_clean":"2023-07-06T02:22:58.546232+0000","last_became_active":"2023-07-06T02:22:52.420089+0000","last_became_peered":"2023-07-06T02:22:52.420089+0000","last_unstale":"2023-07-06T02:22:58.546232+0000","last_undegraded":"2023-07-06T02:22:58.546232+0000","last_fullsized":"2023-07-06T02:22:58.546232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642490+0000","last_change":"2023-07-06T02:22:53.432339+0000","last_active":"2023-07-06T02:22:59.642490+0000","last_peered":"2023-07-06T02:22:59.642490+0000","last_clean":"2023-07-06T02:22:59.642490+0000","last_became_active":"2023-07-06T02:22:53.431993+0000","last_became_peered":"2023-07-06T02:22:53.431993+0000","last_unstale":"2023-07-06T02:22:59.642490+0000","last_undegraded":"2023-07-06T02:22:59.642490+0000","last_fullsized":"2023-07-06T02:22:59.642490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462370+0000","last_change":"2023-07-06T02:22:57.457792+0000","last_active":"2023-07-06T02:22:58.462370+0000","last_peered":"2023-07-06T02:22:58.462370+0000","last_clean":"2023-07-06T02:22:58.462370+0000","last_became_active":"2023-07-06T02:22:57.457396+0000","last_became_peered":"2023-07-06T02:22:57.457396+0000","last_unstale":"2023-07-06T02:22:58.462370+0000","last_undegraded":"2023-07-06T02:22:58.462370+0000","last_fullsized":"2023-07-06T02:22:58.462370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:17.206072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546155+0000","last_change":"2023-07-06T02:22:52.421892+0000","last_active":"2023-07-06T02:22:58.546155+0000","last_peered":"2023-07-06T02:22:58.546155+0000","last_clean":"2023-07-06T02:22:58.546155+0000","last_became_active":"2023-07-06T02:22:52.421613+0000","last_became_peered":"2023-07-06T02:22:52.421613+0000","last_unstale":"2023-07-06T02:22:58.546155+0000","last_undegraded":"2023-07-06T02:22:58.546155+0000","last_fullsized":"2023-07-06T02:22:58.546155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622924+0000","last_change":"2023-07-06T02:22:53.422219+0000","last_active":"2023-07-06T02:22:58.622924+0000","last_peered":"2023-07-06T02:22:58.622924+0000","last_clean":"2023-07-06T02:22:58.622924+0000","last_became_active":"2023-07-06T02:22:53.421934+0000","last_became_peered":"2023-07-06T02:22:53.421934+0000","last_unstale":"2023-07-06T02:22:58.622924+0000","last_undegraded":"2023-07-06T02:22:58.622924+0000","last_fullsized":"2023-07-06T02:22:58.622924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:14:41.243585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462824+0000","last_change":"2023-07-06T02:22:57.450611+0000","last_active":"2023-07-06T02:22:58.462824+0000","last_peered":"2023-07-06T02:22:58.462824+0000","last_clean":"2023-07-06T02:22:58.462824+0000","last_became_active":"2023-07-06T02:22:57.450255+0000","last_became_peered":"2023-07-06T02:22:57.450255+0000","last_unstale":"2023-07-06T02:22:58.462824+0000","last_undegraded":"2023-07-06T02:22:58.462824+0000","last_fullsized":"2023-07-06T02:22:58.462824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:04.461198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545892+0000","last_change":"2023-07-06T02:22:52.420450+0000","last_active":"2023-07-06T02:22:58.545892+0000","last_peered":"2023-07-06T02:22:58.545892+0000","last_clean":"2023-07-06T02:22:58.545892+0000","last_became_active":"2023-07-06T02:22:52.420171+0000","last_became_peered":"2023-07-06T02:22:52.420171+0000","last_unstale":"2023-07-06T02:22:58.545892+0000","last_undegraded":"2023-07-06T02:22:58.545892+0000","last_fullsized":"2023-07-06T02:22:58.545892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:35.849912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462735+0000","last_change":"2023-07-06T02:22:53.425285+0000","last_active":"2023-07-06T02:22:58.462735+0000","last_peered":"2023-07-06T02:22:58.462735+0000","last_clean":"2023-07-06T02:22:58.462735+0000","last_became_active":"2023-07-06T02:22:53.424994+0000","last_became_peered":"2023-07-06T02:22:53.424994+0000","last_unstale":"2023-07-06T02:22:58.462735+0000","last_undegraded":"2023-07-06T02:22:58.462735+0000","last_fullsized":"2023-07-06T02:22:58.462735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.580209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462875+0000","last_change":"2023-07-06T02:22:57.459471+0000","last_active":"2023-07-06T02:22:58.462875+0000","last_peered":"2023-07-06T02:22:58.462875+0000","last_clean":"2023-07-06T02:22:58.462875+0000","last_became_active":"2023-07-06T02:22:57.459086+0000","last_became_peered":"2023-07-06T02:22:57.459086+0000","last_unstale":"2023-07-06T02:22:58.462875+0000","last_undegraded":"2023-07-06T02:22:58.462875+0000","last_fullsized":"2023-07-06T02:22:58.462875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:32.995367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643074+0000","last_change":"2023-07-06T02:22:52.419379+0000","last_active":"2023-07-06T02:22:59.643074+0000","last_peered":"2023-07-06T02:22:59.643074+0000","last_clean":"2023-07-06T02:22:59.643074+0000","last_became_active":"2023-07-06T02:22:52.419100+0000","last_became_peered":"2023-07-06T02:22:52.419100+0000","last_unstale":"2023-07-06T02:22:59.643074+0000","last_undegraded":"2023-07-06T02:22:59.643074+0000","last_fullsized":"2023-07-06T02:22:59.643074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:15.612393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642483+0000","last_change":"2023-07-06T02:22:53.423846+0000","last_active":"2023-07-06T02:22:59.642483+0000","last_peered":"2023-07-06T02:22:59.642483+0000","last_clean":"2023-07-06T02:22:59.642483+0000","last_became_active":"2023-07-06T02:22:53.423585+0000","last_became_peered":"2023-07-06T02:22:53.423585+0000","last_unstale":"2023-07-06T02:22:59.642483+0000","last_undegraded":"2023-07-06T02:22:59.642483+0000","last_fullsized":"2023-07-06T02:22:59.642483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:05.040887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462786+0000","last_change":"2023-07-06T02:22:57.450230+0000","last_active":"2023-07-06T02:22:58.462786+0000","last_peered":"2023-07-06T02:22:58.462786+0000","last_clean":"2023-07-06T02:22:58.462786+0000","last_became_active":"2023-07-06T02:22:57.449841+0000","last_became_peered":"2023-07-06T02:22:57.449841+0000","last_unstale":"2023-07-06T02:22:58.462786+0000","last_undegraded":"2023-07-06T02:22:58.462786+0000","last_fullsized":"2023-07-06T02:22:58.462786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:26:07.959918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462612+0000","last_change":"2023-07-06T02:22:52.418001+0000","last_active":"2023-07-06T02:22:58.462612+0000","last_peered":"2023-07-06T02:22:58.462612+0000","last_clean":"2023-07-06T02:22:58.462612+0000","last_became_active":"2023-07-06T02:22:52.417737+0000","last_became_peered":"2023-07-06T02:22:52.417737+0000","last_unstale":"2023-07-06T02:22:58.462612+0000","last_undegraded":"2023-07-06T02:22:58.462612+0000","last_fullsized":"2023-07-06T02:22:58.462612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623068+0000","last_change":"2023-07-06T02:22:53.418072+0000","last_active":"2023-07-06T02:22:58.623068+0000","last_peered":"2023-07-06T02:22:58.623068+0000","last_clean":"2023-07-06T02:22:58.623068+0000","last_became_active":"2023-07-06T02:22:53.416070+0000","last_became_peered":"2023-07-06T02:22:53.416070+0000","last_unstale":"2023-07-06T02:22:58.623068+0000","last_undegraded":"2023-07-06T02:22:58.623068+0000","last_fullsized":"2023-07-06T02:22:58.623068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:39:55.241653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497345+0000","last_change":"2023-07-06T02:22:57.451350+0000","last_active":"2023-07-06T02:22:58.497345+0000","last_peered":"2023-07-06T02:22:58.497345+0000","last_clean":"2023-07-06T02:22:58.497345+0000","last_became_active":"2023-07-06T02:22:57.450988+0000","last_became_peered":"2023-07-06T02:22:57.450988+0000","last_unstale":"2023-07-06T02:22:58.497345+0000","last_undegraded":"2023-07-06T02:22:58.497345+0000","last_fullsized":"2023-07-06T02:22:58.497345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:08:02.280780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462907+0000","last_change":"2023-07-06T02:22:52.423191+0000","last_active":"2023-07-06T02:22:58.462907+0000","last_peered":"2023-07-06T02:22:58.462907+0000","last_clean":"2023-07-06T02:22:58.462907+0000","last_became_active":"2023-07-06T02:22:52.422863+0000","last_became_peered":"2023-07-06T02:22:52.422863+0000","last_unstale":"2023-07-06T02:22:58.462907+0000","last_undegraded":"2023-07-06T02:22:58.462907+0000","last_fullsized":"2023-07-06T02:22:58.462907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:22.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623436+0000","last_change":"2023-07-06T02:22:53.419350+0000","last_active":"2023-07-06T02:22:58.623436+0000","last_peered":"2023-07-06T02:22:58.623436+0000","last_clean":"2023-07-06T02:22:58.623436+0000","last_became_active":"2023-07-06T02:22:53.419024+0000","last_became_peered":"2023-07-06T02:22:53.419024+0000","last_unstale":"2023-07-06T02:22:58.623436+0000","last_undegraded":"2023-07-06T02:22:58.623436+0000","last_fullsized":"2023-07-06T02:22:58.623436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:33.746001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643344+0000","last_change":"2023-07-06T02:22:57.455212+0000","last_active":"2023-07-06T02:22:59.643344+0000","last_peered":"2023-07-06T02:22:59.643344+0000","last_clean":"2023-07-06T02:22:59.643344+0000","last_became_active":"2023-07-06T02:22:57.454773+0000","last_became_peered":"2023-07-06T02:22:57.454773+0000","last_unstale":"2023-07-06T02:22:59.643344+0000","last_undegraded":"2023-07-06T02:22:59.643344+0000","last_fullsized":"2023-07-06T02:22:59.643344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622698+0000","last_change":"2023-07-06T02:22:52.417326+0000","last_active":"2023-07-06T02:22:58.622698+0000","last_peered":"2023-07-06T02:22:58.622698+0000","last_clean":"2023-07-06T02:22:58.622698+0000","last_became_active":"2023-07-06T02:22:52.417065+0000","last_became_peered":"2023-07-06T02:22:52.417065+0000","last_unstale":"2023-07-06T02:22:58.622698+0000","last_undegraded":"2023-07-06T02:22:58.622698+0000","last_fullsized":"2023-07-06T02:22:58.622698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:35.235033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623747+0000","last_change":"2023-07-06T02:22:53.422511+0000","last_active":"2023-07-06T02:22:58.623747+0000","last_peered":"2023-07-06T02:22:58.623747+0000","last_clean":"2023-07-06T02:22:58.623747+0000","last_became_active":"2023-07-06T02:22:53.420266+0000","last_became_peered":"2023-07-06T02:22:53.420266+0000","last_unstale":"2023-07-06T02:22:58.623747+0000","last_undegraded":"2023-07-06T02:22:58.623747+0000","last_fullsized":"2023-07-06T02:22:58.623747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:05.329078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623647+0000","last_change":"2023-07-06T02:22:57.451156+0000","last_active":"2023-07-06T02:22:58.623647+0000","last_peered":"2023-07-06T02:22:58.623647+0000","last_clean":"2023-07-06T02:22:58.623647+0000","last_became_active":"2023-07-06T02:22:57.450784+0000","last_became_peered":"2023-07-06T02:22:57.450784+0000","last_unstale":"2023-07-06T02:22:58.623647+0000","last_undegraded":"2023-07-06T02:22:58.623647+0000","last_fullsized":"2023-07-06T02:22:58.623647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:15:13.002024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462668+0000","last_change":"2023-07-06T02:22:52.418727+0000","last_active":"2023-07-06T02:22:58.462668+0000","last_peered":"2023-07-06T02:22:58.462668+0000","last_clean":"2023-07-06T02:22:58.462668+0000","last_became_active":"2023-07-06T02:22:52.418454+0000","last_became_peered":"2023-07-06T02:22:52.418454+0000","last_unstale":"2023-07-06T02:22:58.462668+0000","last_undegraded":"2023-07-06T02:22:58.462668+0000","last_fullsized":"2023-07-06T02:22:58.462668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:05.425463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497348+0000","last_change":"2023-07-06T02:22:53.432617+0000","last_active":"2023-07-06T02:22:58.497348+0000","last_peered":"2023-07-06T02:22:58.497348+0000","last_clean":"2023-07-06T02:22:58.497348+0000","last_became_active":"2023-07-06T02:22:53.432332+0000","last_became_peered":"2023-07-06T02:22:53.432332+0000","last_unstale":"2023-07-06T02:22:58.497348+0000","last_undegraded":"2023-07-06T02:22:58.497348+0000","last_fullsized":"2023-07-06T02:22:58.497348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623091+0000","last_change":"2023-07-06T02:22:57.463354+0000","last_active":"2023-07-06T02:22:58.623091+0000","last_peered":"2023-07-06T02:22:58.623091+0000","last_clean":"2023-07-06T02:22:58.623091+0000","last_became_active":"2023-07-06T02:22:57.462934+0000","last_became_peered":"2023-07-06T02:22:57.462934+0000","last_unstale":"2023-07-06T02:22:58.623091+0000","last_undegraded":"2023-07-06T02:22:58.623091+0000","last_fullsized":"2023-07-06T02:22:58.623091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:23.690834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463354+0000","last_change":"2023-07-06T02:22:52.421219+0000","last_active":"2023-07-06T02:22:58.463354+0000","last_peered":"2023-07-06T02:22:58.463354+0000","last_clean":"2023-07-06T02:22:58.463354+0000","last_became_active":"2023-07-06T02:22:52.420927+0000","last_became_peered":"2023-07-06T02:22:52.420927+0000","last_unstale":"2023-07-06T02:22:58.463354+0000","last_undegraded":"2023-07-06T02:22:58.463354+0000","last_fullsized":"2023-07-06T02:22:58.463354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:17:11.011220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623582+0000","last_change":"2023-07-06T02:22:47.973696+0000","last_active":"2023-07-06T02:22:58.623582+0000","last_peered":"2023-07-06T02:22:58.623582+0000","last_clean":"2023-07-06T02:22:58.623582+0000","last_became_active":"2023-07-06T02:22:47.973358+0000","last_became_peered":"2023-07-06T02:22:47.973358+0000","last_unstale":"2023-07-06T02:22:58.623582+0000","last_undegraded":"2023-07-06T02:22:58.623582+0000","last_fullsized":"2023-07-06T02:22:58.623582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:26:55.060583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462551+0000","last_change":"2023-07-06T02:22:53.433182+0000","last_active":"2023-07-06T02:22:58.462551+0000","last_peered":"2023-07-06T02:22:58.462551+0000","last_clean":"2023-07-06T02:22:58.462551+0000","last_became_active":"2023-07-06T02:22:53.432900+0000","last_became_peered":"2023-07-06T02:22:53.432900+0000","last_unstale":"2023-07-06T02:22:58.462551+0000","last_undegraded":"2023-07-06T02:22:58.462551+0000","last_fullsized":"2023-07-06T02:22:58.462551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462417+0000","last_change":"2023-07-06T02:22:57.456985+0000","last_active":"2023-07-06T02:22:58.462417+0000","last_peered":"2023-07-06T02:22:58.462417+0000","last_clean":"2023-07-06T02:22:58.462417+0000","last_became_active":"2023-07-06T02:22:57.456594+0000","last_became_peered":"2023-07-06T02:22:57.456594+0000","last_unstale":"2023-07-06T02:22:58.462417+0000","last_undegraded":"2023-07-06T02:22:58.462417+0000","last_fullsized":"2023-07-06T02:22:58.462417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:44.241938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463270+0000","last_change":"2023-07-06T02:22:47.323237+0000","last_active":"2023-07-06T02:22:58.463270+0000","last_peered":"2023-07-06T02:22:58.463270+0000","last_clean":"2023-07-06T02:22:58.463270+0000","last_became_active":"2023-07-06T02:22:47.322763+0000","last_became_peered":"2023-07-06T02:22:47.322763+0000","last_unstale":"2023-07-06T02:22:58.463270+0000","last_undegraded":"2023-07-06T02:22:58.463270+0000","last_fullsized":"2023-07-06T02:22:58.463270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:51:10.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642236+0000","last_change":"2023-07-06T02:22:52.417097+0000","last_active":"2023-07-06T02:22:59.642236+0000","last_peered":"2023-07-06T02:22:59.642236+0000","last_clean":"2023-07-06T02:22:59.642236+0000","last_became_active":"2023-07-06T02:22:52.416824+0000","last_became_peered":"2023-07-06T02:22:52.416824+0000","last_unstale":"2023-07-06T02:22:59.642236+0000","last_undegraded":"2023-07-06T02:22:59.642236+0000","last_fullsized":"2023-07-06T02:22:59.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:35.415046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622712+0000","last_change":"2023-07-06T02:22:53.419559+0000","last_active":"2023-07-06T02:22:58.622712+0000","last_peered":"2023-07-06T02:22:58.622712+0000","last_clean":"2023-07-06T02:22:58.622712+0000","last_became_active":"2023-07-06T02:22:53.419069+0000","last_became_peered":"2023-07-06T02:22:53.419069+0000","last_unstale":"2023-07-06T02:22:58.622712+0000","last_undegraded":"2023-07-06T02:22:58.622712+0000","last_fullsized":"2023-07-06T02:22:58.622712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:22.045924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643230+0000","last_change":"2023-07-06T02:22:57.459369+0000","last_active":"2023-07-06T02:22:59.643230+0000","last_peered":"2023-07-06T02:22:59.643230+0000","last_clean":"2023-07-06T02:22:59.643230+0000","last_became_active":"2023-07-06T02:22:57.458965+0000","last_became_peered":"2023-07-06T02:22:57.458965+0000","last_unstale":"2023-07-06T02:22:59.643230+0000","last_undegraded":"2023-07-06T02:22:59.643230+0000","last_fullsized":"2023-07-06T02:22:59.643230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:40.450527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496831+0000","last_change":"2023-07-06T02:22:47.325598+0000","last_active":"2023-07-06T02:22:58.496831+0000","last_peered":"2023-07-06T02:22:58.496831+0000","last_clean":"2023-07-06T02:22:58.496831+0000","last_became_active":"2023-07-06T02:22:47.325315+0000","last_became_peered":"2023-07-06T02:22:47.325315+0000","last_unstale":"2023-07-06T02:22:58.496831+0000","last_undegraded":"2023-07-06T02:22:58.496831+0000","last_fullsized":"2023-07-06T02:22:58.496831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:36.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545612+0000","last_change":"2023-07-06T02:22:47.326882+0000","last_active":"2023-07-06T02:22:58.545612+0000","last_peered":"2023-07-06T02:22:58.545612+0000","last_clean":"2023-07-06T02:22:58.545612+0000","last_became_active":"2023-07-06T02:22:47.326511+0000","last_became_peered":"2023-07-06T02:22:47.326511+0000","last_unstale":"2023-07-06T02:22:58.545612+0000","last_undegraded":"2023-07-06T02:22:58.545612+0000","last_fullsized":"2023-07-06T02:22:58.545612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:46.985907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641990+0000","last_change":"2023-07-06T02:22:52.416206+0000","last_active":"2023-07-06T02:22:59.641990+0000","last_peered":"2023-07-06T02:22:59.641990+0000","last_clean":"2023-07-06T02:22:59.641990+0000","last_became_active":"2023-07-06T02:22:52.415905+0000","last_became_peered":"2023-07-06T02:22:52.415905+0000","last_unstale":"2023-07-06T02:22:59.641990+0000","last_undegraded":"2023-07-06T02:22:59.641990+0000","last_fullsized":"2023-07-06T02:22:59.641990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:59.401932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642535+0000","last_change":"2023-07-06T02:22:53.423988+0000","last_active":"2023-07-06T02:22:59.642535+0000","last_peered":"2023-07-06T02:22:59.642535+0000","last_clean":"2023-07-06T02:22:59.642535+0000","last_became_active":"2023-07-06T02:22:53.423711+0000","last_became_peered":"2023-07-06T02:22:53.423711+0000","last_unstale":"2023-07-06T02:22:59.642535+0000","last_undegraded":"2023-07-06T02:22:59.642535+0000","last_fullsized":"2023-07-06T02:22:59.642535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463130+0000","last_change":"2023-07-06T02:22:57.453513+0000","last_active":"2023-07-06T02:22:58.463130+0000","last_peered":"2023-07-06T02:22:58.463130+0000","last_clean":"2023-07-06T02:22:58.463130+0000","last_became_active":"2023-07-06T02:22:57.453163+0000","last_became_peered":"2023-07-06T02:22:57.453163+0000","last_unstale":"2023-07-06T02:22:58.463130+0000","last_undegraded":"2023-07-06T02:22:58.463130+0000","last_fullsized":"2023-07-06T02:22:58.463130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:06:26.324856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545616+0000","last_change":"2023-07-06T02:22:47.613219+0000","last_active":"2023-07-06T02:22:58.545616+0000","last_peered":"2023-07-06T02:22:58.545616+0000","last_clean":"2023-07-06T02:22:58.545616+0000","last_became_active":"2023-07-06T02:22:47.612787+0000","last_became_peered":"2023-07-06T02:22:47.612787+0000","last_unstale":"2023-07-06T02:22:58.545616+0000","last_undegraded":"2023-07-06T02:22:58.545616+0000","last_fullsized":"2023-07-06T02:22:58.545616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:24.728713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496749+0000","last_change":"2023-07-06T02:22:52.421146+0000","last_active":"2023-07-06T02:22:58.496749+0000","last_peered":"2023-07-06T02:22:58.496749+0000","last_clean":"2023-07-06T02:22:58.496749+0000","last_became_active":"2023-07-06T02:22:52.420937+0000","last_became_peered":"2023-07-06T02:22:52.420937+0000","last_unstale":"2023-07-06T02:22:58.496749+0000","last_undegraded":"2023-07-06T02:22:58.496749+0000","last_fullsized":"2023-07-06T02:22:58.496749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:41.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497242+0000","last_change":"2023-07-06T02:22:53.428966+0000","last_active":"2023-07-06T02:22:58.497242+0000","last_peered":"2023-07-06T02:22:58.497242+0000","last_clean":"2023-07-06T02:22:58.497242+0000","last_became_active":"2023-07-06T02:22:53.428644+0000","last_became_peered":"2023-07-06T02:22:53.428644+0000","last_unstale":"2023-07-06T02:22:58.497242+0000","last_undegraded":"2023-07-06T02:22:58.497242+0000","last_fullsized":"2023-07-06T02:22:58.497242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:10.006953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545738+0000","last_change":"2023-07-06T02:22:57.445443+0000","last_active":"2023-07-06T02:22:58.545738+0000","last_peered":"2023-07-06T02:22:58.545738+0000","last_clean":"2023-07-06T02:22:58.545738+0000","last_became_active":"2023-07-06T02:22:57.445042+0000","last_became_peered":"2023-07-06T02:22:57.445042+0000","last_unstale":"2023-07-06T02:22:58.545738+0000","last_undegraded":"2023-07-06T02:22:58.545738+0000","last_fullsized":"2023-07-06T02:22:58.545738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:09.069048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545450+0000","last_change":"2023-07-06T02:22:47.613102+0000","last_active":"2023-07-06T02:22:58.545450+0000","last_peered":"2023-07-06T02:22:58.545450+0000","last_clean":"2023-07-06T02:22:58.545450+0000","last_became_active":"2023-07-06T02:22:47.612655+0000","last_became_peered":"2023-07-06T02:22:47.612655+0000","last_unstale":"2023-07-06T02:22:58.545450+0000","last_undegraded":"2023-07-06T02:22:58.545450+0000","last_fullsized":"2023-07-06T02:22:58.545450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:30.336091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462609+0000","last_change":"2023-07-06T02:22:52.419398+0000","last_active":"2023-07-06T02:22:58.462609+0000","last_peered":"2023-07-06T02:22:58.462609+0000","last_clean":"2023-07-06T02:22:58.462609+0000","last_became_active":"2023-07-06T02:22:52.419139+0000","last_became_peered":"2023-07-06T02:22:52.419139+0000","last_unstale":"2023-07-06T02:22:58.462609+0000","last_undegraded":"2023-07-06T02:22:58.462609+0000","last_fullsized":"2023-07-06T02:22:58.462609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463313+0000","last_change":"2023-07-06T02:22:53.425795+0000","last_active":"2023-07-06T02:22:58.463313+0000","last_peered":"2023-07-06T02:22:58.463313+0000","last_clean":"2023-07-06T02:22:58.463313+0000","last_became_active":"2023-07-06T02:22:53.425518+0000","last_became_peered":"2023-07-06T02:22:53.425518+0000","last_unstale":"2023-07-06T02:22:58.463313+0000","last_undegraded":"2023-07-06T02:22:58.463313+0000","last_fullsized":"2023-07-06T02:22:58.463313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:21:40.141734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642407+0000","last_change":"2023-07-06T02:22:57.457031+0000","last_active":"2023-07-06T02:22:59.642407+0000","last_peered":"2023-07-06T02:22:59.642407+0000","last_clean":"2023-07-06T02:22:59.642407+0000","last_became_active":"2023-07-06T02:22:57.456653+0000","last_became_peered":"2023-07-06T02:22:57.456653+0000","last_unstale":"2023-07-06T02:22:59.642407+0000","last_undegraded":"2023-07-06T02:22:59.642407+0000","last_fullsized":"2023-07-06T02:22:59.642407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:34.843149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642558+0000","last_change":"2023-07-06T02:22:47.613009+0000","last_active":"2023-07-06T02:22:59.642558+0000","last_peered":"2023-07-06T02:22:59.642558+0000","last_clean":"2023-07-06T02:22:59.642558+0000","last_became_active":"2023-07-06T02:22:47.612664+0000","last_became_peered":"2023-07-06T02:22:47.612664+0000","last_unstale":"2023-07-06T02:22:59.642558+0000","last_undegraded":"2023-07-06T02:22:59.642558+0000","last_fullsized":"2023-07-06T02:22:59.642558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:35.690988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623032+0000","last_change":"2023-07-06T02:22:52.417974+0000","last_active":"2023-07-06T02:22:58.623032+0000","last_peered":"2023-07-06T02:22:58.623032+0000","last_clean":"2023-07-06T02:22:58.623032+0000","last_became_active":"2023-07-06T02:22:52.417681+0000","last_became_peered":"2023-07-06T02:22:52.417681+0000","last_unstale":"2023-07-06T02:22:58.623032+0000","last_undegraded":"2023-07-06T02:22:58.623032+0000","last_fullsized":"2023-07-06T02:22:58.623032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:28:34.626494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496776+0000","last_change":"2023-07-06T02:22:53.424117+0000","last_active":"2023-07-06T02:22:58.496776+0000","last_peered":"2023-07-06T02:22:58.496776+0000","last_clean":"2023-07-06T02:22:58.496776+0000","last_became_active":"2023-07-06T02:22:53.423755+0000","last_became_peered":"2023-07-06T02:22:53.423755+0000","last_unstale":"2023-07-06T02:22:58.496776+0000","last_undegraded":"2023-07-06T02:22:58.496776+0000","last_fullsized":"2023-07-06T02:22:58.496776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:48.677462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463117+0000","last_change":"2023-07-06T02:22:57.451130+0000","last_active":"2023-07-06T02:22:58.463117+0000","last_peered":"2023-07-06T02:22:58.463117+0000","last_clean":"2023-07-06T02:22:58.463117+0000","last_became_active":"2023-07-06T02:22:57.450726+0000","last_became_peered":"2023-07-06T02:22:57.450726+0000","last_unstale":"2023-07-06T02:22:58.463117+0000","last_undegraded":"2023-07-06T02:22:58.463117+0000","last_fullsized":"2023-07-06T02:22:58.463117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:38.492011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463258+0000","last_change":"2023-07-06T02:22:47.612949+0000","last_active":"2023-07-06T02:22:58.463258+0000","last_peered":"2023-07-06T02:22:58.463258+0000","last_clean":"2023-07-06T02:22:58.463258+0000","last_became_active":"2023-07-06T02:22:47.612606+0000","last_became_peered":"2023-07-06T02:22:47.612606+0000","last_unstale":"2023-07-06T02:22:58.463258+0000","last_undegraded":"2023-07-06T02:22:58.463258+0000","last_fullsized":"2023-07-06T02:22:58.463258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:26:58.278350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496991+0000","last_change":"2023-07-06T02:22:52.421787+0000","last_active":"2023-07-06T02:22:58.496991+0000","last_peered":"2023-07-06T02:22:58.496991+0000","last_clean":"2023-07-06T02:22:58.496991+0000","last_became_active":"2023-07-06T02:22:52.421536+0000","last_became_peered":"2023-07-06T02:22:52.421536+0000","last_unstale":"2023-07-06T02:22:58.496991+0000","last_undegraded":"2023-07-06T02:22:58.496991+0000","last_fullsized":"2023-07-06T02:22:58.496991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:50.380161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642624+0000","last_change":"2023-07-06T02:22:53.419643+0000","last_active":"2023-07-06T02:22:59.642624+0000","last_peered":"2023-07-06T02:22:59.642624+0000","last_clean":"2023-07-06T02:22:59.642624+0000","last_became_active":"2023-07-06T02:22:53.418153+0000","last_became_peered":"2023-07-06T02:22:53.418153+0000","last_unstale":"2023-07-06T02:22:59.642624+0000","last_undegraded":"2023-07-06T02:22:59.642624+0000","last_fullsized":"2023-07-06T02:22:59.642624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642319+0000","last_change":"2023-07-06T02:22:57.455969+0000","last_active":"2023-07-06T02:22:59.642319+0000","last_peered":"2023-07-06T02:22:59.642319+0000","last_clean":"2023-07-06T02:22:59.642319+0000","last_became_active":"2023-07-06T02:22:57.455566+0000","last_became_peered":"2023-07-06T02:22:57.455566+0000","last_unstale":"2023-07-06T02:22:59.642319+0000","last_undegraded":"2023-07-06T02:22:59.642319+0000","last_fullsized":"2023-07-06T02:22:59.642319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:15.032393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463409+0000","last_change":"2023-07-06T02:22:47.329340+0000","last_active":"2023-07-06T02:22:58.463409+0000","last_peered":"2023-07-06T02:22:58.463409+0000","last_clean":"2023-07-06T02:22:58.463409+0000","last_became_active":"2023-07-06T02:22:47.329090+0000","last_became_peered":"2023-07-06T02:22:47.329090+0000","last_unstale":"2023-07-06T02:22:58.463409+0000","last_undegraded":"2023-07-06T02:22:58.463409+0000","last_fullsized":"2023-07-06T02:22:58.463409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:42.929148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643052+0000","last_change":"2023-07-06T02:22:52.422406+0000","last_active":"2023-07-06T02:22:59.643052+0000","last_peered":"2023-07-06T02:22:59.643052+0000","last_clean":"2023-07-06T02:22:59.643052+0000","last_became_active":"2023-07-06T02:22:52.422125+0000","last_became_peered":"2023-07-06T02:22:52.422125+0000","last_unstale":"2023-07-06T02:22:59.643052+0000","last_undegraded":"2023-07-06T02:22:59.643052+0000","last_fullsized":"2023-07-06T02:22:59.643052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:08.999634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496575+0000","last_change":"2023-07-06T02:22:53.428897+0000","last_active":"2023-07-06T02:22:58.496575+0000","last_peered":"2023-07-06T02:22:58.496575+0000","last_clean":"2023-07-06T02:22:58.496575+0000","last_became_active":"2023-07-06T02:22:53.428651+0000","last_became_peered":"2023-07-06T02:22:53.428651+0000","last_unstale":"2023-07-06T02:22:58.496575+0000","last_undegraded":"2023-07-06T02:22:58.496575+0000","last_fullsized":"2023-07-06T02:22:58.496575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:49.714737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497099+0000","last_change":"2023-07-06T02:22:57.454674+0000","last_active":"2023-07-06T02:22:58.497099+0000","last_peered":"2023-07-06T02:22:58.497099+0000","last_clean":"2023-07-06T02:22:58.497099+0000","last_became_active":"2023-07-06T02:22:57.454162+0000","last_became_peered":"2023-07-06T02:22:57.454162+0000","last_unstale":"2023-07-06T02:22:58.497099+0000","last_undegraded":"2023-07-06T02:22:58.497099+0000","last_fullsized":"2023-07-06T02:22:58.497099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:56.768674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642632+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:59.642632+0000","last_peered":"2023-07-06T02:22:59.642632+0000","last_clean":"2023-07-06T02:22:59.642632+0000","last_became_active":"2023-07-06T02:22:52.418794+0000","last_became_peered":"2023-07-06T02:22:52.418794+0000","last_unstale":"2023-07-06T02:22:59.642632+0000","last_undegraded":"2023-07-06T02:22:59.642632+0000","last_fullsized":"2023-07-06T02:22:59.642632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463505+0000","last_change":"2023-07-06T02:22:53.424846+0000","last_active":"2023-07-06T02:22:58.463505+0000","last_peered":"2023-07-06T02:22:58.463505+0000","last_clean":"2023-07-06T02:22:58.463505+0000","last_became_active":"2023-07-06T02:22:53.424606+0000","last_became_peered":"2023-07-06T02:22:53.424606+0000","last_unstale":"2023-07-06T02:22:58.463505+0000","last_undegraded":"2023-07-06T02:22:58.463505+0000","last_fullsized":"2023-07-06T02:22:58.463505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:31:39.592702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545637+0000","last_change":"2023-07-06T02:22:57.455280+0000","last_active":"2023-07-06T02:22:58.545637+0000","last_peered":"2023-07-06T02:22:58.545637+0000","last_clean":"2023-07-06T02:22:58.545637+0000","last_became_active":"2023-07-06T02:22:57.454860+0000","last_became_peered":"2023-07-06T02:22:57.454860+0000","last_unstale":"2023-07-06T02:22:58.545637+0000","last_undegraded":"2023-07-06T02:22:58.545637+0000","last_fullsized":"2023-07-06T02:22:58.545637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642892+0000","last_change":"2023-07-06T02:22:52.419322+0000","last_active":"2023-07-06T02:22:59.642892+0000","last_peered":"2023-07-06T02:22:59.642892+0000","last_clean":"2023-07-06T02:22:59.642892+0000","last_became_active":"2023-07-06T02:22:52.418950+0000","last_became_peered":"2023-07-06T02:22:52.418950+0000","last_unstale":"2023-07-06T02:22:59.642892+0000","last_undegraded":"2023-07-06T02:22:59.642892+0000","last_fullsized":"2023-07-06T02:22:59.642892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:22.417730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623010+0000","last_change":"2023-07-06T02:22:53.419784+0000","last_active":"2023-07-06T02:22:58.623010+0000","last_peered":"2023-07-06T02:22:58.623010+0000","last_clean":"2023-07-06T02:22:58.623010+0000","last_became_active":"2023-07-06T02:22:53.419477+0000","last_became_peered":"2023-07-06T02:22:53.419477+0000","last_unstale":"2023-07-06T02:22:58.623010+0000","last_undegraded":"2023-07-06T02:22:58.623010+0000","last_fullsized":"2023-07-06T02:22:58.623010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:07.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643049+0000","last_change":"2023-07-06T02:22:57.447138+0000","last_active":"2023-07-06T02:22:59.643049+0000","last_peered":"2023-07-06T02:22:59.643049+0000","last_clean":"2023-07-06T02:22:59.643049+0000","last_became_active":"2023-07-06T02:22:57.446786+0000","last_became_peered":"2023-07-06T02:22:57.446786+0000","last_unstale":"2023-07-06T02:22:59.643049+0000","last_undegraded":"2023-07-06T02:22:59.643049+0000","last_fullsized":"2023-07-06T02:22:59.643049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:45.545737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643069+0000","last_change":"2023-07-06T02:22:52.419733+0000","last_active":"2023-07-06T02:22:59.643069+0000","last_peered":"2023-07-06T02:22:59.643069+0000","last_clean":"2023-07-06T02:22:59.643069+0000","last_became_active":"2023-07-06T02:22:52.418972+0000","last_became_peered":"2023-07-06T02:22:52.418972+0000","last_unstale":"2023-07-06T02:22:59.643069+0000","last_undegraded":"2023-07-06T02:22:59.643069+0000","last_fullsized":"2023-07-06T02:22:59.643069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:07:57.245299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622844+0000","last_change":"2023-07-06T02:22:53.416731+0000","last_active":"2023-07-06T02:22:58.622844+0000","last_peered":"2023-07-06T02:22:58.622844+0000","last_clean":"2023-07-06T02:22:58.622844+0000","last_became_active":"2023-07-06T02:22:53.416412+0000","last_became_peered":"2023-07-06T02:22:53.416412+0000","last_unstale":"2023-07-06T02:22:58.622844+0000","last_undegraded":"2023-07-06T02:22:58.622844+0000","last_fullsized":"2023-07-06T02:22:58.622844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:36:20.268819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622716+0000","last_change":"2023-07-06T02:22:57.461025+0000","last_active":"2023-07-06T02:22:58.622716+0000","last_peered":"2023-07-06T02:22:58.622716+0000","last_clean":"2023-07-06T02:22:58.622716+0000","last_became_active":"2023-07-06T02:22:57.460633+0000","last_became_peered":"2023-07-06T02:22:57.460633+0000","last_unstale":"2023-07-06T02:22:58.622716+0000","last_undegraded":"2023-07-06T02:22:58.622716+0000","last_fullsized":"2023-07-06T02:22:58.622716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:26.873513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:52.418771+0000","last_became_peered":"2023-07-06T02:22:52.418771+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:24:21.889565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463056+0000","last_change":"2023-07-06T02:22:53.421840+0000","last_active":"2023-07-06T02:22:58.463056+0000","last_peered":"2023-07-06T02:22:58.463056+0000","last_clean":"2023-07-06T02:22:58.463056+0000","last_became_active":"2023-07-06T02:22:53.421644+0000","last_became_peered":"2023-07-06T02:22:53.421644+0000","last_unstale":"2023-07-06T02:22:58.463056+0000","last_undegraded":"2023-07-06T02:22:58.463056+0000","last_fullsized":"2023-07-06T02:22:58.463056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:42:56.076946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:57.459681+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:57.459298+0000","last_became_peered":"2023-07-06T02:22:57.459298+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:49.871715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642530+0000","last_change":"2023-07-06T02:22:52.425605+0000","last_active":"2023-07-06T02:22:59.642530+0000","last_peered":"2023-07-06T02:22:59.642530+0000","last_clean":"2023-07-06T02:22:59.642530+0000","last_became_active":"2023-07-06T02:22:52.425345+0000","last_became_peered":"2023-07-06T02:22:52.425345+0000","last_unstale":"2023-07-06T02:22:59.642530+0000","last_undegraded":"2023-07-06T02:22:59.642530+0000","last_fullsized":"2023-07-06T02:22:59.642530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622815+0000","last_change":"2023-07-06T02:22:53.422473+0000","last_active":"2023-07-06T02:22:58.622815+0000","last_peered":"2023-07-06T02:22:58.622815+0000","last_clean":"2023-07-06T02:22:58.622815+0000","last_became_active":"2023-07-06T02:22:53.422171+0000","last_became_peered":"2023-07-06T02:22:53.422171+0000","last_unstale":"2023-07-06T02:22:58.622815+0000","last_undegraded":"2023-07-06T02:22:58.622815+0000","last_fullsized":"2023-07-06T02:22:58.622815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:35.957039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462966+0000","last_change":"2023-07-06T02:22:53.420559+0000","last_active":"2023-07-06T02:22:58.462966+0000","last_peered":"2023-07-06T02:22:58.462966+0000","last_clean":"2023-07-06T02:22:58.462966+0000","last_became_active":"2023-07-06T02:22:53.420220+0000","last_became_peered":"2023-07-06T02:22:53.420220+0000","last_unstale":"2023-07-06T02:22:58.462966+0000","last_undegraded":"2023-07-06T02:22:58.462966+0000","last_fullsized":"2023-07-06T02:22:58.462966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:11.617347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462676+0000","last_change":"2023-07-06T02:22:53.424835+0000","last_active":"2023-07-06T02:22:58.462676+0000","last_peered":"2023-07-06T02:22:58.462676+0000","last_clean":"2023-07-06T02:22:58.462676+0000","last_became_active":"2023-07-06T02:22:53.424520+0000","last_became_peered":"2023-07-06T02:22:53.424520+0000","last_unstale":"2023-07-06T02:22:58.462676+0000","last_undegraded":"2023-07-06T02:22:58.462676+0000","last_fullsized":"2023-07-06T02:22:58.462676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:27.623742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623256+0000","last_change":"2023-07-06T02:22:53.420661+0000","last_active":"2023-07-06T02:22:58.623256+0000","last_peered":"2023-07-06T02:22:58.623256+0000","last_clean":"2023-07-06T02:22:58.623256+0000","last_became_active":"2023-07-06T02:22:53.420137+0000","last_became_peered":"2023-07-06T02:22:53.420137+0000","last_unstale":"2023-07-06T02:22:58.623256+0000","last_undegraded":"2023-07-06T02:22:58.623256+0000","last_fullsized":"2023-07-06T02:22:58.623256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:07:51.965833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462455+0000","last_change":"2023-07-06T02:22:53.434477+0000","last_active":"2023-07-06T02:22:58.462455+0000","last_peered":"2023-07-06T02:22:58.462455+0000","last_clean":"2023-07-06T02:22:58.462455+0000","last_became_active":"2023-07-06T02:22:53.434127+0000","last_became_peered":"2023-07-06T02:22:53.434127+0000","last_unstale":"2023-07-06T02:22:58.462455+0000","last_undegraded":"2023-07-06T02:22:58.462455+0000","last_fullsized":"2023-07-06T02:22:58.462455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:53.433904+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:53.433683+0000","last_became_peered":"2023-07-06T02:22:53.433683+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T02:23:12.657 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T02:23:12.657 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-06T02:23:12.657 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-06T02:23:12.658 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-06T02:23:12.658 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-06T02:23:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.806+0000 7f2a65863700 1 Processor -- start 2023-07-06T02:23:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.806+0000 7f2a65863700 1 -- start start 2023-07-06T02:23:12.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a65863700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a60156ab0 0x7f2a60160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a65863700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a601600b0 0x7f2a600cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a65863700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600cb3d0 con 0x7f2a60160bb0 2023-07-06T02:23:12.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a65863700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a60156e80 con 0x7f2a60156ab0 2023-07-06T02:23:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a65863700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600ca370 con 0x7f2a601600b0 2023-07-06T02:23:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a5e7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2a60160bb0 0x7f2a600b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57406/0 (socket says 172.21.15.89:57406) 2023-07-06T02:23:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a5e7fc700 1 -- 172.21.15.89:0/1645813213 learned_addr learned my addr 172.21.15.89:0/1645813213 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812132109 0 0) 0x7f2a600cb3d0 con 0x7f2a60160bb0 2023-07-06T02:23:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a5effd700 1 --2- 172.21.15.89:0/1645813213 >> v2:172.21.15.172:3300/0 conn(0x7f2a60156ab0 0x7f2a60160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a5f7fe700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a601600b0 0x7f2a600cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.807+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a480035b0 con 0x7f2a60160bb0 2023-07-06T02:23:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1177872778 0 0) 0x7f2a480035b0 con 0x7f2a60160bb0 2023-07-06T02:23:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a601600b0 msgr2=0x7f2a600cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a601600b0 0x7f2a600cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 >> v2:172.21.15.172:3300/0 conn(0x7f2a60156ab0 msgr2=0x7f2a60160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 --2- 172.21.15.89:0/1645813213 >> v2:172.21.15.172:3300/0 conn(0x7f2a60156ab0 0x7f2a60160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600ca610 con 0x7f2a60160bb0 2023-07-06T02:23:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f2a50001670 con 0x7f2a60160bb0 2023-07-06T02:23:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 >> v1:172.21.15.89:6789/0 conn(0x7f2a60160bb0 legacy=0x7f2a600b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a48004430 0x7f2a480048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a48004fc0 0x7f2a4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60156e80 con 0x7f2a48003a00 2023-07-06T02:23:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600ca370 con 0x7f2a48004fc0 2023-07-06T02:23:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4800bf50 con 0x7f2a48004430 2023-07-06T02:23:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5f7fe700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a601600b0 0x7f2a600cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5effd700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5e7fc700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a48004430 0x7f2a480048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5f7fe700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a48004fc0 0x7f2a4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5effd700 1 -- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a48004430 msgr2=0x7f2a480048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5effd700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a48004430 0x7f2a480048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5effd700 1 -- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a48004fc0 msgr2=0x7f2a4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5effd700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a48004fc0 0x7f2a4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5effd700 1 -- 172.21.15.89:0/1645813213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480035b0 con 0x7f2a48003a00 2023-07-06T02:23:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5effd700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009b90 tx=0x7f2a48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec46bbd47ec81dc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400e6c0 con 0x7f2a48003a00 2023-07-06T02:23:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.808+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400e820 con 0x7f2a48003a00 2023-07-06T02:23:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a5dffb700 1 -- 172.21.15.89:0/1645813213 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400eb90 con 0x7f2a48003a00 2023-07-06T02:23:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a65863700 1 -- 172.21.15.89:0/1645813213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 msgr2=0x7f2a48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a65863700 1 --2- 172.21.15.89:0/1645813213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009b90 tx=0x7f2a48005880 comp rx=0 tx=0).stop 2023-07-06T02:23:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a5f7fe700 1 -- 172.21.15.89:0/1645813213 reap_dead start 2023-07-06T02:23:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a65863700 1 -- 172.21.15.89:0/1645813213 shutdown_connections 2023-07-06T02:23:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a65863700 1 -- 172.21.15.89:0/1645813213 >> 172.21.15.89:0/1645813213 conn(0x7f2a600cc490 msgr2=0x7f2a600cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a65863700 1 -- 172.21.15.89:0/1645813213 shutdown_connections 2023-07-06T02:23:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.809+0000 7f2a65863700 1 -- 172.21.15.89:0/1645813213 wait complete. 2023-07-06T02:23:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.810+0000 7f2a65863700 1 Processor -- start 2023-07-06T02:23:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.810+0000 7f2a65863700 1 -- start start 2023-07-06T02:23:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a65863700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a601f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a65863700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b4f60 0x7f2a600b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a65863700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600c4520 0x7f2a600c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a65863700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c4ed0 con 0x7f2a48003a00 2023-07-06T02:23:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a65863700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600b3be0 con 0x7f2a600b4f60 2023-07-06T02:23:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a65863700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600b2860 con 0x7f2a600c4520 2023-07-06T02:23:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a5effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a601f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a5effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a601f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60746/0 (socket says 172.21.15.89:60746) 2023-07-06T02:23:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.811+0000 7f2a5effd700 1 -- 172.21.15.89:0/4020574897 learned_addr learned my addr 172.21.15.89:0/4020574897 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5e7fc700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b4f60 0x7f2a600b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5effd700 1 -- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600c4520 msgr2=0x7f2a600c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5f7fe700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600c4520 0x7f2a600c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5effd700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600c4520 0x7f2a600c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5effd700 1 -- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b4f60 msgr2=0x7f2a600b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5effd700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b4f60 0x7f2a600b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5effd700 1 -- 172.21.15.89:0/4020574897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600b2b40 con 0x7f2a48003a00 2023-07-06T02:23:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5e7fc700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b4f60 0x7f2a600b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5f7fe700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600c4520 0x7f2a600c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a5effd700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a601f5440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a540090b0 tx=0x7f2a540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a47fff700 1 -- 172.21.15.89:0/4020574897 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400e6c0 con 0x7f2a48003a00 2023-07-06T02:23:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a47fff700 1 -- 172.21.15.89:0/4020574897 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a54003ab0 con 0x7f2a48003a00 2023-07-06T02:23:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a600c4ed0 con 0x7f2a48003a00 2023-07-06T02:23:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a47fff700 1 -- 172.21.15.89:0/4020574897 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54003e60 con 0x7f2a48003a00 2023-07-06T02:23:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.812+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600b3100 con 0x7f2a48003a00 2023-07-06T02:23:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.813+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a60074270 con 0x7f2a48003a00 2023-07-06T02:23:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.813+0000 7f2a47fff700 1 -- 172.21.15.89:0/4020574897 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a54013470 con 0x7f2a48003a00 2023-07-06T02:23:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.814+0000 7f2a47fff700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4c06ace0 0x7f2a4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.814+0000 7f2a5e7fc700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4c06ace0 0x7f2a4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.814+0000 7f2a47fff700 1 -- 172.21.15.89:0/4020574897 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2a5408af80 con 0x7f2a48003a00 2023-07-06T02:23:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.814+0000 7f2a5e7fc700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4c06ace0 0x7f2a4c06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a600c5ff0 tx=0x7f2a50000ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:12.818+0000 7f2a47fff700 1 -- 172.21.15.89:0/4020574897 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a540bd310 con 0x7f2a48003a00 2023-07-06T02:23:13.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.063+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f2a6006e5f0 con 0x7f2a48003a00 2023-07-06T02:23:13.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.066+0000 7f2a47fff700 1 -- 172.21.15.89:0/4020574897 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170841 (secure 0 0 0) 0x7f2a540bbb70 con 0x7f2a48003a00 2023-07-06T02:23:13.070 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4c06ace0 msgr2=0x7f2a4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4c06ace0 0x7f2a4c06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a600c5ff0 tx=0x7f2a50000ef0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 msgr2=0x7f2a601f5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a601f5440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a540090b0 tx=0x7f2a540047c0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 shutdown_connections 2023-07-06T02:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a600c4520 0x7f2a600c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2a4c06ace0 0x7f2a4c06d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a48003a00 0x7f2a601f5440 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 --2- 172.21.15.89:0/4020574897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600b4f60 0x7f2a600b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 >> 172.21.15.89:0/4020574897 conn(0x7f2a600cc490 msgr2=0x7f2a6015db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 shutdown_connections 2023-07-06T02:23:13.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.068+0000 7f2a65863700 1 -- 172.21.15.89:0/4020574897 wait complete. 2023-07-06T02:23:13.083 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3591115711},{"type":"v1","addr":"172.21.15.89:6833","nonce":3591115711}]},"active_addr":"172.21.15.89:6833/3591115711","active_change":"2023-07-06T02:22:39.967057+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1963486121}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":994438293}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1852698508}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3240964866}]}]} 2023-07-06T02:23:13.088 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-06T02:23:13.088 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-06T02:23:13.089 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.207+0000 7f0896feb700 1 Processor -- start 2023-07-06T02:23:13.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.207+0000 7f0896feb700 1 -- start start 2023-07-06T02:23:13.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0896feb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 0x7f08900b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0896feb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f089006bfa0 0x7f089006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0896feb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08900cab50 con 0x7f089015c690 2023-07-06T02:23:13.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0896feb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089015de50 con 0x7f089006bfa0 2023-07-06T02:23:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0896feb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900b1e20 con 0x7f08900b1490 2023-07-06T02:23:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0894d87700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f089015c690 0x7f08900cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57424/0 (socket says 172.21.15.89:57424) 2023-07-06T02:23:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0894d87700 1 -- 172.21.15.89:0/1570656063 learned_addr learned my addr 172.21.15.89:0/1570656063 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473055602 0 0) 0x7f08900cab50 con 0x7f089015c690 2023-07-06T02:23:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f0895588700 1 --2- 172.21.15.89:0/1570656063 >> v2:172.21.15.172:3300/0 conn(0x7f089006bfa0 0x7f089006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088ffff700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 0x7f08900b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08800035b0 con 0x7f089015c690 2023-07-06T02:23:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 338165443 0 0) 0x7f08800035b0 con 0x7f089015c690 2023-07-06T02:23:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 msgr2=0x7f08900b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 0x7f08900b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 >> v2:172.21.15.172:3300/0 conn(0x7f089006bfa0 msgr2=0x7f089006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 --2- 172.21.15.89:0/1570656063 >> v2:172.21.15.172:3300/0 conn(0x7f089006bfa0 0x7f089006c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089006c920 con 0x7f089015c690 2023-07-06T02:23:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f0884001670 con 0x7f089015c690 2023-07-06T02:23:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.208+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 >> v1:172.21.15.89:6789/0 conn(0x7f089015c690 legacy=0x7f08900cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 0x7f0880003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0880003f80 0x7f0880004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0880004d90 0x7f0880005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089015de50 con 0x7f0880004d90 2023-07-06T02:23:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900b1e20 con 0x7f0880003f80 2023-07-06T02:23:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088000ba10 con 0x7f08900b1490 2023-07-06T02:23:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0895588700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0880004d90 0x7f0880005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0894d87700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 0x7f0880003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088ffff700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0880003f80 0x7f0880004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0895588700 1 -- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 msgr2=0x7f0880003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0895588700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08900b1490 0x7f0880003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0895588700 1 -- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0880003f80 msgr2=0x7f0880004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0895588700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0880003f80 0x7f0880004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0895588700 1 -- 172.21.15.89:0/1570656063 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800035b0 con 0x7f0880004d90 2023-07-06T02:23:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f0895588700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0880004d90 0x7f0880005200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0888009fc0 tx=0x7f0880005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a339a1372c2a7384 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088800ec30 con 0x7f0880004d90 2023-07-06T02:23:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088800f040 con 0x7f0880004d90 2023-07-06T02:23:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.209+0000 7f088f7fe700 1 -- 172.21.15.89:0/1570656063 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888014480 con 0x7f0880004d90 2023-07-06T02:23:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.210+0000 7f0896feb700 1 -- 172.21.15.89:0/1570656063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0880004d90 msgr2=0x7f0880005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.210+0000 7f0896feb700 1 --2- 172.21.15.89:0/1570656063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0880004d90 0x7f0880005200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0888009fc0 tx=0x7f0880005880 comp rx=0 tx=0).stop 2023-07-06T02:23:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.210+0000 7f0895588700 1 -- 172.21.15.89:0/1570656063 reap_dead start 2023-07-06T02:23:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.210+0000 7f0896feb700 1 -- 172.21.15.89:0/1570656063 shutdown_connections 2023-07-06T02:23:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.210+0000 7f0896feb700 1 -- 172.21.15.89:0/1570656063 >> 172.21.15.89:0/1570656063 conn(0x7f08900cc490 msgr2=0x7f08900cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.210+0000 7f0896feb700 1 -- 172.21.15.89:0/1570656063 shutdown_connections 2023-07-06T02:23:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.210+0000 7f0896feb700 1 -- 172.21.15.89:0/1570656063 wait complete. 2023-07-06T02:23:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.211+0000 7f0896feb700 1 Processor -- start 2023-07-06T02:23:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.211+0000 7f0896feb700 1 -- start start 2023-07-06T02:23:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0896feb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890159c90 0x7f089006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0896feb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f089015a060 0x7f089006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0896feb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f089015ae30 0x7f089015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0896feb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089015b7e0 con 0x7f0890159c90 2023-07-06T02:23:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0896feb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900a7df0 con 0x7f089015a060 2023-07-06T02:23:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0896feb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900a8070 con 0x7f089015ae30 2023-07-06T02:23:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890159c90 0x7f089006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890159c90 0x7f089006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60766/0 (socket says 172.21.15.89:60766) 2023-07-06T02:23:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 -- 172.21.15.89:0/2752025297 learned_addr learned my addr 172.21.15.89:0/2752025297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0895588700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f089015ae30 0x7f089015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f088ffff700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f089015a060 0x7f089006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 -- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f089015ae30 msgr2=0x7f089015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f089015ae30 0x7f089015b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 -- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f089015a060 msgr2=0x7f089006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f089015a060 0x7f089006e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 -- 172.21.15.89:0/2752025297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900a8350 con 0x7f0890159c90 2023-07-06T02:23:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0894d87700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890159c90 0x7f089006dcf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08840048d0 tx=0x7f088400f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f088d7fa700 1 -- 172.21.15.89:0/2752025297 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884007d00 con 0x7f0890159c90 2023-07-06T02:23:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f088d7fa700 1 -- 172.21.15.89:0/2752025297 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088400fa90 con 0x7f0890159c90 2023-07-06T02:23:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.212+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08900a8630 con 0x7f0890159c90 2023-07-06T02:23:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.213+0000 7f088d7fa700 1 -- 172.21.15.89:0/2752025297 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400fe40 con 0x7f0890159c90 2023-07-06T02:23:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.213+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08900a8bf0 con 0x7f0890159c90 2023-07-06T02:23:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.214+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08900a8db0 con 0x7f0890159c90 2023-07-06T02:23:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.214+0000 7f088d7fa700 1 -- 172.21.15.89:0/2752025297 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f088401d460 con 0x7f0890159c90 2023-07-06T02:23:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.214+0000 7f088d7fa700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.214+0000 7f088d7fa700 1 -- 172.21.15.89:0/2752025297 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f08840898d0 con 0x7f0890159c90 2023-07-06T02:23:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.214+0000 7f088ffff700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.215+0000 7f088ffff700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f087406ace0 0x7f087406d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08900a7800 tx=0x7f087c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.219+0000 7f088d7fa700 1 -- 172.21.15.89:0/2752025297 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0884054690 con 0x7f0890159c90 2023-07-06T02:23:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.406+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0890074270 con 0x7f0890159c90 2023-07-06T02:23:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.407+0000 7f088d7fa700 1 -- 172.21.15.89:0/2752025297 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f0884057cb0 con 0x7f0890159c90 2023-07-06T02:23:13.410 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:13.411 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f087406ace0 msgr2=0x7f087406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f087406ace0 0x7f087406d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08900a7800 tx=0x7f087c008040 comp rx=0 tx=0).stop 2023-07-06T02:23:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890159c90 msgr2=0x7f089006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890159c90 0x7f089006dcf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08840048d0 tx=0x7f088400f4b0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 shutdown_connections 2023-07-06T02:23:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f089015ae30 0x7f089015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890159c90 0x7f089006dcf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 --2- 172.21.15.89:0/2752025297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f089015a060 0x7f089006e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 >> 172.21.15.89:0/2752025297 conn(0x7f08900cc490 msgr2=0x7f08900cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 shutdown_connections 2023-07-06T02:23:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.409+0000 7f0896feb700 1 -- 172.21.15.89:0/2752025297 wait complete. 2023-07-06T02:23:13.425 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-06T02:23:13.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:13.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.546+0000 7f1d880fb700 1 Processor -- start 2023-07-06T02:23:13.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.546+0000 7f1d880fb700 1 -- start start 2023-07-06T02:23:13.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d880fb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d8006bdb0 0x7f1d8006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d880fb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d800cd240 con 0x7f1d800ceda0 2023-07-06T02:23:13.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d880fb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d800cf280 con 0x7f1d8006b610 2023-07-06T02:23:13.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d880fb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d800cd490 con 0x7f1d8006bdb0 2023-07-06T02:23:13.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d86698700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1d800ceda0 0x7f1d8015c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57448/0 (socket says 172.21.15.89:57448) 2023-07-06T02:23:13.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d86698700 1 -- 172.21.15.89:0/1749982804 learned_addr learned my addr 172.21.15.89:0/1749982804 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:13.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d85e97700 1 --2- 172.21.15.89:0/1749982804 >> v2:172.21.15.172:3300/0 conn(0x7f1d8006bdb0 0x7f1d8006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d84e95700 1 -- 172.21.15.89:0/1749982804 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723065062 0 0) 0x7f1d800cd240 con 0x7f1d800ceda0 2023-07-06T02:23:13.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d84e95700 1 -- 172.21.15.89:0/1749982804 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d6c0035b0 con 0x7f1d800ceda0 2023-07-06T02:23:13.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d84e95700 1 -- 172.21.15.89:0/1749982804 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735858449 0 0) 0x7f1d6c0035b0 con 0x7f1d800ceda0 2023-07-06T02:23:13.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d85e97700 1 -- 172.21.15.89:0/1749982804 >> v1:172.21.15.172:6789/0 conn(0x7f1d8006b610 legacy=0x7f1d8006c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d85e97700 1 -- 172.21.15.89:0/1749982804 >> v1:172.21.15.89:6789/0 conn(0x7f1d800ceda0 legacy=0x7f1d8015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.547+0000 7f1d85e97700 1 -- 172.21.15.89:0/1749982804 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800cd770 con 0x7f1d8006bdb0 2023-07-06T02:23:13.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d85e97700 1 --2- 172.21.15.89:0/1749982804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8006bdb0 0x7f1d8006aa70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009840 tx=0x7f1d7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aad745db8afc4061 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d84e95700 1 -- 172.21.15.89:0/1749982804 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780019f0 con 0x7f1d8006bdb0 2023-07-06T02:23:13.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d84e95700 1 -- 172.21.15.89:0/1749982804 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c004030 con 0x7f1d8006bdb0 2023-07-06T02:23:13.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d84e95700 1 -- 172.21.15.89:0/1749982804 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c0043a0 con 0x7f1d8006bdb0 2023-07-06T02:23:13.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d880fb700 1 -- 172.21.15.89:0/1749982804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8006bdb0 msgr2=0x7f1d8006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d880fb700 1 --2- 172.21.15.89:0/1749982804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8006bdb0 0x7f1d8006aa70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009840 tx=0x7f1d7c009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:13.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d880fb700 1 -- 172.21.15.89:0/1749982804 shutdown_connections 2023-07-06T02:23:13.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d880fb700 1 --2- 172.21.15.89:0/1749982804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8006bdb0 0x7f1d8006aa70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d880fb700 1 -- 172.21.15.89:0/1749982804 >> 172.21.15.89:0/1749982804 conn(0x7f1d800b5390 msgr2=0x7f1d800d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d880fb700 1 -- 172.21.15.89:0/1749982804 shutdown_connections 2023-07-06T02:23:13.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.548+0000 7f1d880fb700 1 -- 172.21.15.89:0/1749982804 wait complete. 2023-07-06T02:23:13.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.549+0000 7f1d880fb700 1 Processor -- start 2023-07-06T02:23:13.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d880fb700 1 -- start start 2023-07-06T02:23:13.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d880fb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8006bdb0 0x7f1d800b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d880fb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d800ceda0 0x7f1d800c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d880fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d800cc380 0x7f1d8013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d880fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800c95a0 con 0x7f1d800cc380 2023-07-06T02:23:13.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d880fb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800b62e0 con 0x7f1d800ceda0 2023-07-06T02:23:13.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d880fb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d801f1eb0 con 0x7f1d8006bdb0 2023-07-06T02:23:13.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d86698700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d800cc380 0x7f1d8013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d86698700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d800cc380 0x7f1d8013ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60782/0 (socket says 172.21.15.89:60782) 2023-07-06T02:23:13.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.550+0000 7f1d86698700 1 -- 172.21.15.89:0/4120067107 learned_addr learned my addr 172.21.15.89:0/4120067107 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:13.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d85e97700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8006bdb0 0x7f1d800b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d85696700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d800ceda0 0x7f1d800c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d86698700 1 -- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8006bdb0 msgr2=0x7f1d800b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d86698700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8006bdb0 0x7f1d800b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d86698700 1 -- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d800ceda0 msgr2=0x7f1d800c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d86698700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d800ceda0 0x7f1d800c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d86698700 1 -- 172.21.15.89:0/4120067107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0094a0 con 0x7f1d800cc380 2023-07-06T02:23:13.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d86698700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d800cc380 0x7f1d8013ed40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800cbe0 tx=0x7f1d7800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d767fc700 1 -- 172.21.15.89:0/4120067107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800ab80 con 0x7f1d800cc380 2023-07-06T02:23:13.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d767fc700 1 -- 172.21.15.89:0/4120067107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7800ad00 con 0x7f1d800cc380 2023-07-06T02:23:13.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d801526a0 con 0x7f1d800cc380 2023-07-06T02:23:13.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d767fc700 1 -- 172.21.15.89:0/4120067107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78007310 con 0x7f1d800cc380 2023-07-06T02:23:13.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.551+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d800cb8a0 con 0x7f1d800cc380 2023-07-06T02:23:13.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.552+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d800cff50 con 0x7f1d800cc380 2023-07-06T02:23:13.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.552+0000 7f1d767fc700 1 -- 172.21.15.89:0/4120067107 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d78007470 con 0x7f1d800cc380 2023-07-06T02:23:13.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.553+0000 7f1d767fc700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1d6406ace0 0x7f1d6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.553+0000 7f1d767fc700 1 -- 172.21.15.89:0/4120067107 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1d78089670 con 0x7f1d800cc380 2023-07-06T02:23:13.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.553+0000 7f1d85e97700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1d6406ace0 0x7f1d6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.553+0000 7f1d85e97700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1d6406ace0 0x7f1d6406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0090b0 tx=0x7f1d7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.557+0000 7f1d767fc700 1 -- 172.21.15.89:0/4120067107 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d78054430 con 0x7f1d800cc380 2023-07-06T02:23:13.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.744+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d80074270 con 0x7f1d800cc380 2023-07-06T02:23:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.746+0000 7f1d767fc700 1 -- 172.21.15.89:0/4120067107 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f1d78057a50 con 0x7f1d800cc380 2023-07-06T02:23:13.749 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:13.749 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1d6406ace0 msgr2=0x7f1d6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1d6406ace0 0x7f1d6406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0090b0 tx=0x7f1d7c008030 comp rx=0 tx=0).stop 2023-07-06T02:23:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d800cc380 msgr2=0x7f1d8013ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d800cc380 0x7f1d8013ed40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800cbe0 tx=0x7f1d7800a060 comp rx=0 tx=0).stop 2023-07-06T02:23:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 shutdown_connections 2023-07-06T02:23:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1d6406ace0 0x7f1d6406d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d800cc380 0x7f1d8013ed40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8006bdb0 0x7f1d800b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 --2- 172.21.15.89:0/4120067107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d800ceda0 0x7f1d800c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 >> 172.21.15.89:0/4120067107 conn(0x7f1d800b5390 msgr2=0x7f1d8015c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 shutdown_connections 2023-07-06T02:23:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.748+0000 7f1d880fb700 1 -- 172.21.15.89:0/4120067107 wait complete. 2023-07-06T02:23:13.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-06T02:23:13.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-06T02:23:13.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-06T02:23:13.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-06T02:23:13.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-06T02:23:13.767 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-06T02:23:13.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-06T02:23:13.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-06T02:23:13.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.970+0000 7f4ef8999700 1 Processor -- start 2023-07-06T02:23:13.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.970+0000 7f4ef8999700 1 -- start start 2023-07-06T02:23:13.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.971+0000 7f4ef8999700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ef4163fa0 0x7f4ef40cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.971+0000 7f4ef8999700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ef40af7e0 0x7f4ef40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.971+0000 7f4ef8999700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef4070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.971+0000 7f4ef8999700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef40b0310 con 0x7f4ef4163fa0 2023-07-06T02:23:13.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.971+0000 7f4ef8999700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef40afd50 con 0x7f4ef40af7e0 2023-07-06T02:23:13.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.971+0000 7f4ef8999700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40b0030 con 0x7f4ef40b58d0 2023-07-06T02:23:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.972+0000 7f4ef1d9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ef40af7e0 0x7f4ef40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.972+0000 7f4ef1d9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ef40af7e0 0x7f4ef40b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59546/0 (socket says 172.21.15.89:59546) 2023-07-06T02:23:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.972+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/4074217202 learned_addr learned my addr 172.21.15.89:0/4074217202 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef2d9d700 1 --2- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 msgr2=0x7f4ef4070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:13.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef1d9b700 1 --2- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef4070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/4074217202 >> v2:172.21.15.89:3300/0 conn(0x7f4ef4163fa0 msgr2=0x7f4ef40cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:13.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef1d9b700 1 --2- 172.21.15.89:0/4074217202 >> v2:172.21.15.89:3300/0 conn(0x7f4ef4163fa0 0x7f4ef40cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/4074217202 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef40cb1f0 con 0x7f4ef40af7e0 2023-07-06T02:23:13.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef1d9b700 1 --2- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef40af7e0 0x7f4ef40b5390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4072260 tx=0x7f4ee4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f32247b8d2a0f70 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef159a700 1 -- 172.21.15.89:0/4074217202 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400b870 con 0x7f4ef40af7e0 2023-07-06T02:23:13.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef159a700 1 -- 172.21.15.89:0/4074217202 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee400b9d0 con 0x7f4ef40af7e0 2023-07-06T02:23:13.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.973+0000 7f4ef159a700 1 -- 172.21.15.89:0/4074217202 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400bd60 con 0x7f4ef40af7e0 2023-07-06T02:23:13.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.974+0000 7f4ef8999700 1 -- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef40af7e0 msgr2=0x7f4ef40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.974+0000 7f4ef8999700 1 --2- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef40af7e0 0x7f4ef40b5390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4072260 tx=0x7f4ee4009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.975+0000 7fee43cf2700 1 Processor -- start 2023-07-06T02:23:13.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.975+0000 7fee43cf2700 1 -- start start 2023-07-06T02:23:13.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.975+0000 7f4ef8999700 1 -- 172.21.15.89:0/4074217202 shutdown_connections 2023-07-06T02:23:13.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.975+0000 7f4ef8999700 1 --2- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef4070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.975+0000 7f4ef8999700 1 --2- 172.21.15.89:0/4074217202 >> v2:172.21.15.89:3300/0 conn(0x7f4ef4163fa0 0x7f4ef40cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.975+0000 7f4ef8999700 1 --2- 172.21.15.89:0/4074217202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef40af7e0 0x7f4ef40b5390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.975+0000 7f4ef8999700 1 -- 172.21.15.89:0/4074217202 >> 172.21.15.89:0/4074217202 conn(0x7f4ef40d0b00 msgr2=0x7f4ef40d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee43cf2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fee3c070f20 0x7fee3c071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee43cf2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c0cd480 con 0x7fee3c0cd910 2023-07-06T02:23:13.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee43cf2700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c0718b0 con 0x7fee3c0c2840 2023-07-06T02:23:13.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee43cf2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c06a770 con 0x7fee3c070f20 2023-07-06T02:23:13.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee4128d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fee3c070f20 0x7fee3c071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee4128d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fee3c070f20 0x7fee3c071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59554/0 (socket says 172.21.15.89:59554) 2023-07-06T02:23:13.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee4128d700 1 -- 172.21.15.89:0/776868638 learned_addr learned my addr 172.21.15.89:0/776868638 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:13.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.976+0000 7fee41a8e700 1 --1- 172.21.15.89:0/776868638 >> v1:172.21.15.172:6789/0 conn(0x7fee3c0c2840 0x7fee3c06a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33220/0 (socket says 172.21.15.89:33220) 2023-07-06T02:23:13.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.977+0000 7fee4128d700 1 -- 172.21.15.89:0/776868638 >> v1:172.21.15.172:6789/0 conn(0x7fee3c0c2840 legacy=0x7fee3c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.977+0000 7fee4128d700 1 -- 172.21.15.89:0/776868638 >> v1:172.21.15.89:6789/0 conn(0x7fee3c0cd910 legacy=0x7fee3c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.977+0000 7fee4128d700 1 -- 172.21.15.89:0/776868638 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c06ef60 con 0x7fee3c070f20 2023-07-06T02:23:13.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.977+0000 7fee4128d700 1 --2- 172.21.15.89:0/776868638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c070f20 0x7fee3c071370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0098f0 tx=0x7fee2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb50139d5f5c823 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.977+0000 7fee40a8c700 1 -- 172.21.15.89:0/776868638 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c004230 con 0x7fee3c070f20 2023-07-06T02:23:13.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.977+0000 7fee40a8c700 1 -- 172.21.15.89:0/776868638 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c004390 con 0x7fee3c070f20 2023-07-06T02:23:13.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.977+0000 7fee40a8c700 1 -- 172.21.15.89:0/776868638 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0036a0 con 0x7fee3c070f20 2023-07-06T02:23:13.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.978+0000 7fee43cf2700 1 -- 172.21.15.89:0/776868638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c070f20 msgr2=0x7fee3c071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.978+0000 7fee43cf2700 1 --2- 172.21.15.89:0/776868638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c070f20 0x7fee3c071370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0098f0 tx=0x7fee2c009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:13.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.978+0000 7fee43cf2700 1 -- 172.21.15.89:0/776868638 shutdown_connections 2023-07-06T02:23:13.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.978+0000 7fee43cf2700 1 --2- 172.21.15.89:0/776868638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c070f20 0x7fee3c071370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.978+0000 7fee43cf2700 1 -- 172.21.15.89:0/776868638 >> 172.21.15.89:0/776868638 conn(0x7fee3c0cf1c0 msgr2=0x7fee3c0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.979+0000 7fd53ae38700 1 Processor -- start 2023-07-06T02:23:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.979+0000 7f4ef8999700 1 -- 172.21.15.89:0/4074217202 shutdown_connections 2023-07-06T02:23:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.979+0000 7fee43cf2700 1 -- 172.21.15.89:0/776868638 shutdown_connections 2023-07-06T02:23:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.979+0000 7fee43cf2700 1 -- 172.21.15.89:0/776868638 wait complete. 2023-07-06T02:23:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.980+0000 7f2e45a46700 1 Processor -- start 2023-07-06T02:23:13.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.979+0000 7fd53ae38700 1 -- start start 2023-07-06T02:23:13.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.980+0000 7fd53ae38700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5340cd700 0x7fd534164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.980+0000 7fd53ae38700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340cdb70 con 0x7fd5340aed40 2023-07-06T02:23:13.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.980+0000 7fd53ae38700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340cde00 con 0x7fd5340b1490 2023-07-06T02:23:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.980+0000 7fd53ae38700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5340ce0e0 con 0x7fd5340cd700 2023-07-06T02:23:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7fd538bd4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd5340aed40 0x7fd5340ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57470/0 (socket says 172.21.15.89:57470) 2023-07-06T02:23:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7fd538bd4700 1 -- 172.21.15.89:0/87565941 learned_addr learned my addr 172.21.15.89:0/87565941 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.980+0000 7f2e45a46700 1 -- start start 2023-07-06T02:23:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7f2e45a46700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e400b1ba0 0x7f2e40164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7f2e45a46700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e400cdb70 con 0x7f2e400cd700 2023-07-06T02:23:13.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7f2e45a46700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e400cde00 con 0x7f2e400aed40 2023-07-06T02:23:13.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7f2e45a46700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400ce0e0 con 0x7f2e400b1ba0 2023-07-06T02:23:13.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.979+0000 7f4ef8999700 1 -- 172.21.15.89:0/4074217202 wait complete. 2023-07-06T02:23:13.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7f4ef8999700 1 Processor -- start 2023-07-06T02:23:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7fb2609ca700 1 Processor -- start 2023-07-06T02:23:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.981+0000 7fee43cf2700 1 Processor -- start 2023-07-06T02:23:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fd533fff700 1 --2- 172.21.15.89:0/87565941 >> v2:172.21.15.89:3300/0 conn(0x7fd5340cd700 0x7fd534164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fd533fff700 1 -- 172.21.15.89:0/87565941 >> v1:172.21.15.172:6789/0 conn(0x7fd5340b1490 legacy=0x7fd5340b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fd533fff700 1 -- 172.21.15.89:0/87565941 >> v1:172.21.15.89:6789/0 conn(0x7fd5340aed40 legacy=0x7fd5340ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fd5337fe700 1 -- 172.21.15.89:0/87565941 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932221624 0 0) 0x7fd5340cdb70 con 0x7fd5340aed40 2023-07-06T02:23:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fd533fff700 1 -- 172.21.15.89:0/87565941 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340ce3c0 con 0x7fd5340cd700 2023-07-06T02:23:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fd533fff700 1 --2- 172.21.15.89:0/87565941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd534164e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009d90 tx=0x7fd52c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278943962b43f736 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fd5337fe700 1 -- 172.21.15.89:0/87565941 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00e070 con 0x7fd5340cd700 2023-07-06T02:23:13.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fd5337fe700 1 -- 172.21.15.89:0/87565941 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c00bb90 con 0x7fd5340cd700 2023-07-06T02:23:13.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fd5337fe700 1 -- 172.21.15.89:0/87565941 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c003a20 con 0x7fd5340cd700 2023-07-06T02:23:13.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7f4ef8999700 1 -- start start 2023-07-06T02:23:13.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7f4ef8999700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ef40af7e0 0x7f4ef40c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7f4ef8999700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7f4ef8999700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4163fa0 0x7f4ef41efca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7f4ef8999700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41f6210 con 0x7f4ef40af7e0 2023-07-06T02:23:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7f4ef8999700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef406f640 con 0x7f4ef4163fa0 2023-07-06T02:23:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7f4ef8999700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41f6440 con 0x7f4ef40b58d0 2023-07-06T02:23:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7f2e3f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e400b1ba0 0x7f2e40164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7f2e3f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e400b1ba0 0x7f2e40164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60806/0 (socket says 172.21.15.89:60806) 2023-07-06T02:23:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3670377424 learned_addr learned my addr 172.21.15.89:0/3670377424 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fb2609ca700 1 -- start start 2023-07-06T02:23:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb2609ca700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb25c0c9bb0 0x7fb25c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb2609ca700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb25c0c1f40 0x7fb25c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb2609ca700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c06a060 con 0x7fb25c1608d0 2023-07-06T02:23:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb2609ca700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c06a340 con 0x7fb25c0c1f40 2023-07-06T02:23:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb2609ca700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c06cb60 con 0x7fb25c0c9bb0 2023-07-06T02:23:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb25a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb25c1608d0 0x7fb25c0c94a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57484/0 (socket says 172.21.15.89:57484) 2023-07-06T02:23:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb25a59c700 1 -- 172.21.15.89:0/2893652899 learned_addr learned my addr 172.21.15.89:0/2893652899 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.983+0000 7fb25ad9d700 1 --2- 172.21.15.89:0/2893652899 >> v2:172.21.15.89:3300/0 conn(0x7fb25c0c1f40 0x7fb25c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fd53ae38700 1 -- 172.21.15.89:0/87565941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 msgr2=0x7fd534164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fd53ae38700 1 --2- 172.21.15.89:0/87565941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd534164e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009d90 tx=0x7fd52c00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fd53ae38700 1 -- 172.21.15.89:0/87565941 shutdown_connections 2023-07-06T02:23:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fd53ae38700 1 --2- 172.21.15.89:0/87565941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd534164e80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fd53ae38700 1 -- 172.21.15.89:0/87565941 >> 172.21.15.89:0/87565941 conn(0x7fd5340cc490 msgr2=0x7fd5340cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef406eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55786/0 (socket says 172.21.15.89:55786) 2023-07-06T02:23:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/167529713 learned_addr learned my addr 172.21.15.89:0/167529713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef2d9d700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4163fa0 0x7f4ef41efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fd53ae38700 1 -- 172.21.15.89:0/87565941 shutdown_connections 2023-07-06T02:23:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fd53ae38700 1 -- 172.21.15.89:0/87565941 wait complete. 2023-07-06T02:23:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.982+0000 7fee43cf2700 1 -- start start 2023-07-06T02:23:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee43cf2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee3c070f20 0x7fee3c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee43cf2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c0c2840 0x7fee3c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee43cf2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee3c0cd910 0x7fee3c0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee43cf2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c15f310 con 0x7fee3c070f20 2023-07-06T02:23:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee43cf2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c15ed50 con 0x7fee3c0c2840 2023-07-06T02:23:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee43cf2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c15f030 con 0x7fee3c0cd910 2023-07-06T02:23:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fb25ad9d700 1 -- 172.21.15.89:0/2893652899 >> v2:172.21.15.172:3300/0 conn(0x7fb25c0c9bb0 msgr2=0x7fb25c0cd220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fb25ad9d700 1 --2- 172.21.15.89:0/2893652899 >> v2:172.21.15.172:3300/0 conn(0x7fb25c0c9bb0 0x7fb25c0cd220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fb25ad9d700 1 -- 172.21.15.89:0/2893652899 >> v1:172.21.15.89:6789/0 conn(0x7fb25c1608d0 legacy=0x7fb25c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fb25ad9d700 1 -- 172.21.15.89:0/2893652899 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c0cd840 con 0x7fb25c0c1f40 2023-07-06T02:23:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee4228f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee3c0cd910 0x7fee3c0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee4228f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee3c0cd910 0x7fee3c0c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55804/0 (socket says 172.21.15.89:55804) 2023-07-06T02:23:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee4228f700 1 -- 172.21.15.89:0/2164654905 learned_addr learned my addr 172.21.15.89:0/2164654905 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7fee4128d700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c0c2840 0x7fee3c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef259c700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ef40af7e0 0x7f4ef40c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3670377424 >> v1:172.21.15.172:6789/0 conn(0x7f2e400aed40 legacy=0x7f2e400ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3670377424 >> v1:172.21.15.89:6789/0 conn(0x7f2e400cd700 legacy=0x7f2e400b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3670377424 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400ce3c0 con 0x7f2e400b1ba0 2023-07-06T02:23:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/167529713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4163fa0 msgr2=0x7f4ef41efca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4163fa0 0x7f4ef41efca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ef40af7e0 msgr2=0x7f4ef40c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ef40af7e0 0x7f4ef40c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.984+0000 7f4ef1d9b700 1 -- 172.21.15.89:0/167529713 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4009550 con 0x7f4ef40b58d0 2023-07-06T02:23:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7f4ef259c700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ef40af7e0 0x7f4ef40c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fee41a8e700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee3c070f20 0x7fee3c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fee4228f700 1 -- 172.21.15.89:0/2164654905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c0c2840 msgr2=0x7fee3c15b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fee4228f700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee3c0c2840 0x7fee3c15b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fee4228f700 1 -- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee3c070f20 msgr2=0x7fee3c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fee4228f700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee3c070f20 0x7fee3c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fee4228f700 1 -- 172.21.15.89:0/2164654905 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c009550 con 0x7fee3c0cd910 2023-07-06T02:23:14.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7f2e3f7fe700 1 --2- 172.21.15.89:0/3670377424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400b1ba0 0x7f2e40164e80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e300098f0 tx=0x7f2e30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cdf30a92e85e8a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fee4228f700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee3c0cd910 0x7fee3c0c59f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c450 tx=0x7fee3800c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7fb25ad9d700 1 --2- 172.21.15.89:0/2893652899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c1f40 0x7fb25c06a7d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0c1cd0 tx=0x7fb250009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524c1a7cbb8f7d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7f2e3dffb700 1 -- 172.21.15.89:0/3670377424 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30004230 con 0x7f2e400b1ba0 2023-07-06T02:23:14.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7f2e3dffb700 1 -- 172.21.15.89:0/3670377424 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e30004390 con 0x7f2e400b1ba0 2023-07-06T02:23:14.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.985+0000 7f2e3dffb700 1 -- 172.21.15.89:0/3670377424 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e300036a0 con 0x7f2e400b1ba0 2023-07-06T02:23:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fd53ae38700 1 Processor -- start 2023-07-06T02:23:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7f4ef1d9b700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef406eb60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40afe90 tx=0x7f4ee4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fb25959a700 1 -- 172.21.15.89:0/2893652899 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250011830 con 0x7fb25c0c1f40 2023-07-06T02:23:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fb25959a700 1 -- 172.21.15.89:0/2893652899 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb250011990 con 0x7fb25c0c1f40 2023-07-06T02:23:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fb25959a700 1 -- 172.21.15.89:0/2893652899 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250011d20 con 0x7fb25c0c1f40 2023-07-06T02:23:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380054e0 con 0x7fee3c0cd910 2023-07-06T02:23:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee38003680 con 0x7fee3c0cd910 2023-07-06T02:23:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7f2e45a46700 1 -- 172.21.15.89:0/3670377424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400b1ba0 msgr2=0x7f2e40164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7f2e45a46700 1 --2- 172.21.15.89:0/3670377424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400b1ba0 0x7f2e40164e80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e300098f0 tx=0x7f2e30009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f2e45a46700 1 -- 172.21.15.89:0/3670377424 shutdown_connections 2023-07-06T02:23:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f2e45a46700 1 --2- 172.21.15.89:0/3670377424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400b1ba0 0x7f2e40164e80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f2e45a46700 1 -- 172.21.15.89:0/3670377424 >> 172.21.15.89:0/3670377424 conn(0x7f2e400cc490 msgr2=0x7f2e400cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004440 con 0x7f4ef40b58d0 2023-07-06T02:23:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fb2609ca700 1 -- 172.21.15.89:0/2893652899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c1f40 msgr2=0x7fb25c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fb2609ca700 1 --2- 172.21.15.89:0/2893652899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c1f40 0x7fb25c06a7d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0c1cd0 tx=0x7fb250009b20 comp rx=0 tx=0).stop 2023-07-06T02:23:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee40045a0 con 0x7f4ef40b58d0 2023-07-06T02:23:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40038a0 con 0x7f4ef40b58d0 2023-07-06T02:23:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f2e45a46700 1 -- 172.21.15.89:0/3670377424 shutdown_connections 2023-07-06T02:23:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef41f6720 con 0x7f4ef40b58d0 2023-07-06T02:23:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38003a30 con 0x7fee3c0cd910 2023-07-06T02:23:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c1f6b50 con 0x7fee3c0cd910 2023-07-06T02:23:14.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c15b930 con 0x7fee3c0cd910 2023-07-06T02:23:14.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee24000fc0 con 0x7fee3c0cd910 2023-07-06T02:23:14.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f2e45a46700 1 -- 172.21.15.89:0/3670377424 wait complete. 2023-07-06T02:23:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.986+0000 7fd53ae38700 1 -- start start 2023-07-06T02:23:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd53ae38700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5340aed40 0x7fd53415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd53ae38700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5340b1490 0x7fd534163550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd53ae38700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd53406c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd53ae38700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340b4370 con 0x7fd5340cd700 2023-07-06T02:23:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd53ae38700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340b3a40 con 0x7fd5340aed40 2023-07-06T02:23:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd53ae38700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340b3d20 con 0x7fd5340b1490 2023-07-06T02:23:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd5393d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd53406c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd5393d5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd53406c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60830/0 (socket says 172.21.15.89:60830) 2023-07-06T02:23:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7fd5393d5700 1 -- 172.21.15.89:0/3574279936 learned_addr learned my addr 172.21.15.89:0/3574279936 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fb2609ca700 1 -- 172.21.15.89:0/2893652899 shutdown_connections 2023-07-06T02:23:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fb2609ca700 1 --2- 172.21.15.89:0/2893652899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c1f40 0x7fb25c06a7d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fb2609ca700 1 --2- 172.21.15.89:0/2893652899 >> v2:172.21.15.172:3300/0 conn(0x7fb25c0c9bb0 0x7fb25c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fb2609ca700 1 -- 172.21.15.89:0/2893652899 >> 172.21.15.89:0/2893652899 conn(0x7fb25c0cf8d0 msgr2=0x7fb25c0cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fd5393d5700 1 -- 172.21.15.89:0/3574279936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5340b1490 msgr2=0x7fd534163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fd5393d5700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5340b1490 0x7fd534163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fd5393d5700 1 -- 172.21.15.89:0/3574279936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5340aed40 msgr2=0x7fd53415b180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T02:23:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fd5393d5700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5340aed40 0x7fd53415b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fd5393d5700 1 -- 172.21.15.89:0/3574279936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c009a70 con 0x7fd5340cd700 2023-07-06T02:23:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fb2609ca700 1 -- 172.21.15.89:0/2893652899 shutdown_connections 2023-07-06T02:23:14.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7f59a7f94700 1 Processor -- start 2023-07-06T02:23:14.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.987+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef41fa660 con 0x7f4ef40b58d0 2023-07-06T02:23:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7f2e45a46700 1 Processor -- start 2023-07-06T02:23:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee38003b90 con 0x7fee3c0cd910 2023-07-06T02:23:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fee327fc700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fee2006adb0 0x7fee2006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fee38088f70 con 0x7fee3c0cd910 2023-07-06T02:23:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fee327fc700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6808/3689514906,v1:172.21.15.89:6809/3689514906] conn(0x7fee20074a80 0x7fee20076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 --> [v2:172.21.15.89:6808/3689514906,v1:172.21.15.89:6809/3689514906] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee20077580 con 0x7fee20074a80 2023-07-06T02:23:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee38089320 con 0x7fee3c0cd910 2023-07-06T02:23:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fb2609ca700 1 -- 172.21.15.89:0/2893652899 wait complete. 2023-07-06T02:23:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7fb2609ca700 1 Processor -- start 2023-07-06T02:23:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7f59a7f94700 1 -- start start 2023-07-06T02:23:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f59a7f94700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f59a00aed40 0x7f59a00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f59a7f94700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a01652f0 con 0x7f59a00b1490 2023-07-06T02:23:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f59a7f94700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a00cde00 con 0x7f59a0164e80 2023-07-06T02:23:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f59a7f94700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a00ce0e0 con 0x7f59a00aed40 2023-07-06T02:23:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7f2e45a46700 1 -- start start 2023-07-06T02:23:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f2e45a46700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400aed40 0x7f2e401fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f2e45a46700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e400b1ba0 0x7f2e400b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f2e45a46700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e400cd700 0x7f2e400c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f2e45a46700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c4110 con 0x7f2e400aed40 2023-07-06T02:23:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f2e45a46700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c43f0 con 0x7f2e400cd700 2023-07-06T02:23:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f2e45a46700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c46d0 con 0x7f2e400b1ba0 2023-07-06T02:23:14.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f59a6531700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f59a00b1490 0x7f59a00b1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57508/0 (socket says 172.21.15.89:57508) 2023-07-06T02:23:14.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.990+0000 7f59a6531700 1 -- 172.21.15.89:0/1225035227 learned_addr learned my addr 172.21.15.89:0/1225035227 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:14.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fd5393d5700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd53406c4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd52800c450 tx=0x7fd52800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.988+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011790 con 0x7fd5340cd700 2023-07-06T02:23:14.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ed8000fc0 con 0x7f4ef40b58d0 2023-07-06T02:23:14.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f59a5d30700 1 --2- 172.21.15.89:0/1225035227 >> v2:172.21.15.89:3300/0 conn(0x7f59a00aed40 0x7f59a00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7fee41a8e700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fee2006adb0 0x7fee2006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e400cd700 0x7f2e400c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e400cd700 0x7f2e400c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59602/0 (socket says 172.21.15.89:59602) 2023-07-06T02:23:14.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3805513073 learned_addr learned my addr 172.21.15.89:0/3805513073 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3effd700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400aed40 0x7f2e401fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3e7fc700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e400b1ba0 0x7f2e400b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5280118f0 con 0x7fd5340cd700 2023-07-06T02:23:14.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011ca0 con 0x7fd5340cd700 2023-07-06T02:23:14.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5340b4000 con 0x7fd5340cd700 2023-07-06T02:23:14.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.989+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd534163a90 con 0x7fd5340cd700 2023-07-06T02:23:14.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5340b3bb0 con 0x7fd5340cd700 2023-07-06T02:23:14.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ee4003a00 con 0x7f4ef40b58d0 2023-07-06T02:23:14.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f4ee2ffd700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ed406adb0 0x7f4ed406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f4ef259c700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ed406adb0 0x7f4ed406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f59a5d30700 1 -- 172.21.15.89:0/1225035227 >> v1:172.21.15.172:6789/0 conn(0x7f59a0164e80 legacy=0x7f59a00ceea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:14.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f59a5d30700 1 -- 172.21.15.89:0/1225035227 >> v1:172.21.15.89:6789/0 conn(0x7f59a00b1490 legacy=0x7f59a00b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f59a5d30700 1 -- 172.21.15.89:0/1225035227 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00ce3c0 con 0x7f59a00aed40 2023-07-06T02:23:14.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7f59a5d30700 1 --2- 172.21.15.89:0/1225035227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00aed40 0x7f59a00cd700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f599c009840 tx=0x7f599c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7657e14cadb348e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7fee4128d700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6808/3689514906,v1:172.21.15.89:6809/3689514906] conn(0x7fee20074a80 0x7fee20076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e400b1ba0 msgr2=0x7f2e400b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e400b1ba0 0x7f2e400b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400aed40 msgr2=0x7f2e401fe8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e400aed40 0x7f2e401fe8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.991+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3805513073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30009550 con 0x7f2e400cd700 2023-07-06T02:23:14.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7fee41a8e700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fee2006adb0 0x7fee2006d260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0c9e10 tx=0x7fee34001c50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7f59a4d2e700 1 -- 172.21.15.89:0/1225035227 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c004230 con 0x7f59a00aed40 2023-07-06T02:23:14.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7f59a4d2e700 1 -- 172.21.15.89:0/1225035227 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c004390 con 0x7f59a00aed40 2023-07-06T02:23:14.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7f59a4d2e700 1 -- 172.21.15.89:0/1225035227 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c0036a0 con 0x7f59a00aed40 2023-07-06T02:23:14.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7fb2609ca700 1 -- start start 2023-07-06T02:23:14.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd52801c460 con 0x7fd5340cd700 2023-07-06T02:23:14.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7fee4128d700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6808/3689514906,v1:172.21.15.89:6809/3689514906] conn(0x7fee20074a80 0x7fee20076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7fd5317fa700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.992+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd528089350 con 0x7fd5340cd700 2023-07-06T02:23:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fd5317fa700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6800/990084407,v1:172.21.15.89:6801/990084407] conn(0x7fd5200748b0 0x7fd520076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 --> [v2:172.21.15.89:6800/990084407,v1:172.21.15.89:6801/990084407] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5200773b0 con 0x7fd5200748b0 2023-07-06T02:23:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fd538bd4700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd528054110 con 0x7fd5340cd700 2023-07-06T02:23:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fd533fff700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6800/990084407,v1:172.21.15.89:6801/990084407] conn(0x7fd5200748b0 0x7fd520076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f59a7f94700 1 -- 172.21.15.89:0/1225035227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00aed40 msgr2=0x7f59a00cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f59a7f94700 1 --2- 172.21.15.89:0/1225035227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00aed40 0x7f59a00cd700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f599c009840 tx=0x7f599c009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fec180d7700 1 Processor -- start 2023-07-06T02:23:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f59a7f94700 1 -- 172.21.15.89:0/1225035227 shutdown_connections 2023-07-06T02:23:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f59a7f94700 1 --2- 172.21.15.89:0/1225035227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00aed40 0x7f59a00cd700 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f59a7f94700 1 -- 172.21.15.89:0/1225035227 >> 172.21.15.89:0/1225035227 conn(0x7f59a00cc490 msgr2=0x7f59a00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb2609ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c0c1f40 0x7fb25c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb2609ca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c9bb0 0x7fb25c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb2609ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb25c1608d0 0x7fb25c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb2609ca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b2d20 con 0x7fb25c0c9bb0 2023-07-06T02:23:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb2609ca700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b2110 con 0x7fb25c0c1f40 2023-07-06T02:23:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb2609ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b23f0 con 0x7fb25c1608d0 2023-07-06T02:23:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fd538bd4700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd52006ace0 0x7fd52006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd524007fd0 tx=0x7fd524007920 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fd533fff700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6800/990084407,v1:172.21.15.89:6801/990084407] conn(0x7fd5200748b0 0x7fd520076ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb259d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c9bb0 0x7fb25c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb259d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c9bb0 0x7fb25c0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60858/0 (socket says 172.21.15.89:60858) 2023-07-06T02:23:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb259d9b700 1 -- 172.21.15.89:0/3298223525 learned_addr learned my addr 172.21.15.89:0/3298223525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb25ad9d700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb25c1608d0 0x7fb25c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7fb25a59c700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c0c1f40 0x7fb25c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4ee408ace0 con 0x7f4ef40b58d0 2023-07-06T02:23:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f4ee2ffd700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6816/3305610859,v1:172.21.15.89:6817/3305610859] conn(0x7f4ed4074a00 0x7f4ed4076e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.993+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 --> [v2:172.21.15.89:6816/3305610859,v1:172.21.15.89:6817/3305610859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ed4077500 con 0x7f4ed4074a00 2023-07-06T02:23:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7f4ef259c700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ed406adb0 0x7f4ed406d260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ca8c0 tx=0x7f4ee8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7f2e3f7fe700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e400cd700 0x7f2e400c6f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e4006f2b0 tx=0x7f2e30004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7f4ef2d9d700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6816/3305610859,v1:172.21.15.89:6817/3305610859] conn(0x7f4ed4074a00 0x7f4ed4076e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee4090070 con 0x7f4ef40b58d0 2023-07-06T02:23:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== osd.3 v2:172.21.15.89:6808/3689514906 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fee20077580 con 0x7fee20074a80 2023-07-06T02:23:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb25ad9d700 1 -- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c0c1f40 msgr2=0x7fb25c0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb25ad9d700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c0c1f40 0x7fb25c0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb25ad9d700 1 -- 172.21.15.89:0/3298223525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c9bb0 msgr2=0x7fb25c0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb25ad9d700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb25c0c9bb0 0x7fb25c0b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb25ad9d700 1 -- 172.21.15.89:0/3298223525 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500094a0 con 0x7fb25c1608d0 2023-07-06T02:23:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fec180d7700 1 -- start start 2023-07-06T02:23:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7f4ef2d9d700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6816/3305610859,v1:172.21.15.89:6817/3305610859] conn(0x7f4ed4074a00 0x7f4ed4076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7f59a7f94700 1 -- 172.21.15.89:0/1225035227 shutdown_connections 2023-07-06T02:23:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb25ad9d700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb25c1608d0 0x7fb25c150860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb250016000 tx=0x7fb250004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb24affd700 1 -- 172.21.15.89:0/3298223525 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500036a0 con 0x7fb25c1608d0 2023-07-06T02:23:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb24affd700 1 -- 172.21.15.89:0/3298223525 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25001e070 con 0x7fb25c1608d0 2023-07-06T02:23:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb24affd700 1 -- 172.21.15.89:0/3298223525 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250003a40 con 0x7fb25c1608d0 2023-07-06T02:23:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c0b26d0 con 0x7fb25c1608d0 2023-07-06T02:23:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.994+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c0b4140 con 0x7fb25c1608d0 2023-07-06T02:23:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== osd.0 v2:172.21.15.89:6800/990084407 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd5200773b0 con 0x7fd5200748b0 2023-07-06T02:23:14.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7f59a7f94700 1 -- 172.21.15.89:0/1225035227 wait complete. 2023-07-06T02:23:14.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec180d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006b610 0x7fec1006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec180d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fec100ceda0 0x7fec1015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec180d7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec100cd240 con 0x7fec1006bdb0 2023-07-06T02:23:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec180d7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec100cf280 con 0x7fec100ceda0 2023-07-06T02:23:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec180d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec100cd490 con 0x7fec1006b610 2023-07-06T02:23:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec16674700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fec100ceda0 0x7fec1015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30011630 con 0x7f2e400cd700 2023-07-06T02:23:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== osd.1 v2:172.21.15.89:6816/3305610859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4ed4077500 con 0x7f4ed4074a00 2023-07-06T02:23:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec16674700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fec100ceda0 0x7fec1015c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60866/0 (socket says 172.21.15.89:60866) 2023-07-06T02:23:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7fec16674700 1 -- 172.21.15.89:0/3438601645 learned_addr learned my addr 172.21.15.89:0/3438601645 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.996+0000 7fec16674700 1 -- 172.21.15.89:0/3438601645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006b610 msgr2=0x7fec1006aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:23:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.996+0000 7fec16674700 1 --2- 172.21.15.89:0/3438601645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006b610 0x7fec1006aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.996+0000 7fec16674700 1 -- 172.21.15.89:0/3438601645 >> v1:172.21.15.89:6789/0 conn(0x7fec1006bdb0 legacy=0x7fec1006c180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:23:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.996+0000 7fec16674700 1 -- 172.21.15.89:0/3438601645 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec100cd770 con 0x7fec100ceda0 2023-07-06T02:23:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.996+0000 7fec16674700 1 --2- 172.21.15.89:0/3438601645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec1015c8b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fec08009e50 tx=0x7fec0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5e0a65088421b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e30011790 con 0x7f2e400cd700 2023-07-06T02:23:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.995+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30011b20 con 0x7f2e400cd700 2023-07-06T02:23:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.996+0000 7fec14e71700 1 -- 172.21.15.89:0/3438601645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0800e070 con 0x7fec100ceda0 2023-07-06T02:23:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.996+0000 7fec14e71700 1 -- 172.21.15.89:0/3438601645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec08003a20 con 0x7fec100ceda0 2023-07-06T02:23:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7fec14e71700 1 -- 172.21.15.89:0/3438601645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08003dd0 con 0x7fec100ceda0 2023-07-06T02:23:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7fb24affd700 1 -- 172.21.15.89:0/3298223525 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb250003ba0 con 0x7fb25c1608d0 2023-07-06T02:23:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7f59a7f94700 1 Processor -- start 2023-07-06T02:23:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7fb24affd700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb23c06ace0 0x7fb23c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7fec180d7700 1 -- 172.21.15.89:0/3438601645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 msgr2=0x7fec1015c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7fec180d7700 1 --2- 172.21.15.89:0/3438601645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec1015c8b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fec08009e50 tx=0x7fec0800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7fb24affd700 1 -- 172.21.15.89:0/3298223525 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb25008a6e0 con 0x7fb25c1608d0 2023-07-06T02:23:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7fb25a59c700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb23c06ace0 0x7fb23c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7fb25a59c700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb23c06ace0 0x7fb23c06d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005910 tx=0x7fb24c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.997+0000 7f59a7f94700 1 -- start start 2023-07-06T02:23:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a7f94700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59a00aed40 0x7f59a015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a7f94700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00b1490 0x7f59a0163900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a7f94700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59a00c8170 0x7f59a0201a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a7f94700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a015c280 con 0x7f59a00b1490 2023-07-06T02:23:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a7f94700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00c9770 con 0x7f59a00c8170 2023-07-06T02:23:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a7f94700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00c8b60 con 0x7f59a00aed40 2023-07-06T02:23:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7fec180d7700 1 -- 172.21.15.89:0/3438601645 shutdown_connections 2023-07-06T02:23:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7fec180d7700 1 --2- 172.21.15.89:0/3438601645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec1015c8b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7fec180d7700 1 --2- 172.21.15.89:0/3438601645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006b610 0x7fec1006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7fec180d7700 1 -- 172.21.15.89:0/3438601645 >> 172.21.15.89:0/3438601645 conn(0x7fec100b5390 msgr2=0x7fec100d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a6531700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59a00c8170 0x7f59a0201a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a5d30700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59a00aed40 0x7f59a015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a6531700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59a00c8170 0x7f59a0201a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59630/0 (socket says 172.21.15.89:59630) 2023-07-06T02:23:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a6531700 1 -- 172.21.15.89:0/1855358744 learned_addr learned my addr 172.21.15.89:0/1855358744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a5d30700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59a00aed40 0x7f59a015b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55846/0 (socket says 172.21.15.89:55846) 2023-07-06T02:23:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.998+0000 7f59a552f700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00b1490 0x7f59a0163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7fec180d7700 1 -- 172.21.15.89:0/3438601645 shutdown_connections 2023-07-06T02:23:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e400c49b0 con 0x7f2e400cd700 2023-07-06T02:23:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e401fee20 con 0x7f2e400cd700 2023-07-06T02:23:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f59a6531700 1 -- 172.21.15.89:0/1855358744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59a00aed40 msgr2=0x7f59a015b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f59a6531700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59a00aed40 0x7f59a015b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f59a6531700 1 -- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00b1490 msgr2=0x7f59a0163900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f59a6531700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00b1490 0x7f59a0163900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f59a6531700 1 -- 172.21.15.89:0/1855358744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c0094a0 con 0x7f59a00c8170 2023-07-06T02:23:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f59a552f700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a00b1490 0x7f59a0163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7fec180d7700 1 -- 172.21.15.89:0/3438601645 wait complete. 2023-07-06T02:23:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7fb2609ca700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:6820/466302387,v1:172.21.15.172:6822/466302387] conn(0x7fb25c150da0 0x7fb25c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 --> [v2:172.21.15.172:6820/466302387,v1:172.21.15.172:6822/466302387] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb25c0c2880 con 0x7fb25c150da0 2023-07-06T02:23:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:13.999+0000 7f59a6531700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59a00c8170 0x7f59a0201a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f599800cca0 tx=0x7f599800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.000+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e400ce980 con 0x7f2e400cd700 2023-07-06T02:23:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.000+0000 7f59967fc700 1 -- 172.21.15.89:0/1855358744 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599800ab30 con 0x7f59a00c8170 2023-07-06T02:23:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.000+0000 7f59967fc700 1 -- 172.21.15.89:0/1855358744 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599800ac90 con 0x7f59a00c8170 2023-07-06T02:23:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.000+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a00c8e40 con 0x7f59a00c8170 2023-07-06T02:23:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.000+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a00c9400 con 0x7f59a00c8170 2023-07-06T02:23:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.000+0000 7f59967fc700 1 -- 172.21.15.89:0/1855358744 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980070d0 con 0x7f59a00c8170 2023-07-06T02:23:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.000+0000 7fb259d9b700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:6820/466302387,v1:172.21.15.172:6822/466302387] conn(0x7fb25c150da0 0x7fb25c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.001+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e30022070 con 0x7f2e400cd700 2023-07-06T02:23:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.001+0000 7fec180d7700 1 Processor -- start 2023-07-06T02:23:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.001+0000 7f2e27fff700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2e2c06ad30 0x7f2e2c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.001+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2e3008b3c0 con 0x7f2e400cd700 2023-07-06T02:23:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.001+0000 7f2e27fff700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:6808/2248870656,v1:172.21.15.172:6809/2248870656] conn(0x7f2e2c074a00 0x7f2e2c076e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 --> [v2:172.21.15.172:6808/2248870656,v1:172.21.15.172:6809/2248870656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e2c077500 con 0x7f2e2c074a00 2023-07-06T02:23:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e3008b860 con 0x7f2e400cd700 2023-07-06T02:23:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fb259d9b700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:6820/466302387,v1:172.21.15.172:6822/466302387] conn(0x7fb25c150da0 0x7fb25c06db70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7f59967fc700 1 -- 172.21.15.89:0/1855358744 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5998007230 con 0x7f59a00c8170 2023-07-06T02:23:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7f59967fc700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f598406ad30 0x7f598406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7f59967fc700 1 -- 172.21.15.89:0/1855358744 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5998089590 con 0x7f59a00c8170 2023-07-06T02:23:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.001+0000 7fec180d7700 1 -- start start 2023-07-06T02:23:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fec180d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006bdb0 0x7fec101f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fec180d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec101f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fec180d7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100cc2f0 0x7fec100cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fec180d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100caf70 con 0x7fec100ceda0 2023-07-06T02:23:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fec180d7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100c6a10 con 0x7fec100cc2f0 2023-07-06T02:23:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fec180d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec100c6cf0 con 0x7fec1006bdb0 2023-07-06T02:23:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.002+0000 7fb24affd700 1 -- 172.21.15.89:0/3298223525 <== osd.6 v2:172.21.15.172:6820/466302387 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb25c0c2880 con 0x7fb25c150da0 2023-07-06T02:23:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec101f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15e73700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006bdb0 0x7fec101f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec101f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60880/0 (socket says 172.21.15.89:60880) 2023-07-06T02:23:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 -- 172.21.15.89:0/1821441900 learned_addr learned my addr 172.21.15.89:0/1821441900 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec16674700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100cc2f0 0x7fec100cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 -- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006bdb0 msgr2=0x7fec101f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec1006bdb0 0x7fec101f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 -- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100cc2f0 msgr2=0x7fec100cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec100cc2f0 0x7fec100cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 -- 172.21.15.89:0/1821441900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08009b30 con 0x7fec100ceda0 2023-07-06T02:23:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec15672700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec101f32c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00b450 tx=0x7fec0c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00be60 con 0x7fec100ceda0 2023-07-06T02:23:14.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c011630 con 0x7fec100ceda0 2023-07-06T02:23:14.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec100c6fd0 con 0x7fec100ceda0 2023-07-06T02:23:14.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.003+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0119e0 con 0x7fec100ceda0 2023-07-06T02:23:14.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.004+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec100c6700 con 0x7fec100ceda0 2023-07-06T02:23:14.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.005+0000 7f2e3e7fc700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:6808/2248870656,v1:172.21.15.172:6809/2248870656] conn(0x7f2e2c074a00 0x7f2e2c076e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.005+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec100b2280 con 0x7fec100ceda0 2023-07-06T02:23:14.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.005+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec0c011b40 con 0x7fec100ceda0 2023-07-06T02:23:14.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.005+0000 7f2e3e7fc700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:6808/2248870656,v1:172.21.15.172:6809/2248870656] conn(0x7f2e2c074a00 0x7f2e2c076e30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.005+0000 7fec067fc700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febf406acb0 0x7febf406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec0c089290 con 0x7fec100ceda0 2023-07-06T02:23:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec15e73700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febf406acb0 0x7febf406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec067fc700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:6800/1682295071,v1:172.21.15.172:6801/1682295071] conn(0x7febf4074900 0x7febf4076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 --> [v2:172.21.15.172:6800/1682295071,v1:172.21.15.172:6801/1682295071] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febf4077400 con 0x7febf4074900 2023-07-06T02:23:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec0c08b950 con 0x7fec100ceda0 2023-07-06T02:23:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec16674700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:6800/1682295071,v1:172.21.15.172:6801/1682295071] conn(0x7febf4074900 0x7febf4076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec15e73700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febf406acb0 0x7febf406d160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec00006fd0 tx=0x7fec00009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== osd.5 v2:172.21.15.172:6808/2248870656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2e2c077500 con 0x7f2e2c074a00 2023-07-06T02:23:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.006+0000 7fec16674700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:6800/1682295071,v1:172.21.15.172:6801/1682295071] conn(0x7febf4074900 0x7febf4076d30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.007+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== osd.4 v2:172.21.15.172:6800/1682295071 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7febf4077400 con 0x7febf4074900 2023-07-06T02:23:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.010+0000 7f59a5d30700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f598406ad30 0x7f598406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.010+0000 7f59a5d30700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f598406ad30 0x7f598406d1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f599c0090b0 tx=0x7f599c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.008+0000 7f2e3effd700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2e2c06ad30 0x7f2e2c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.008+0000 7f2e3effd700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2e2c06ad30 0x7f2e2c06d1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800f300 tx=0x7f2e380042c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.014+0000 7f59a7f94700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6820/1124333878,v1:172.21.15.89:6822/1124333878] conn(0x7f59880015e0 0x7f5988003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.014+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 --> [v2:172.21.15.89:6820/1124333878,v1:172.21.15.89:6822/1124333878] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5988006bc0 con 0x7f59880015e0 2023-07-06T02:23:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.014+0000 7f59a552f700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6820/1124333878,v1:172.21.15.89:6822/1124333878] conn(0x7f59880015e0 0x7f5988003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.015+0000 7f59a552f700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6820/1124333878,v1:172.21.15.89:6822/1124333878] conn(0x7f59880015e0 0x7f5988003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.014+0000 7f4668eff700 1 Processor -- start 2023-07-06T02:23:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.014+0000 7f4668eff700 1 -- start start 2023-07-06T02:23:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.016+0000 7f59967fc700 1 -- 172.21.15.89:0/1855358744 <== osd.2 v2:172.21.15.89:6820/1124333878 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5988006bc0 con 0x7f59880015e0 2023-07-06T02:23:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.015+0000 7f4668eff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f46640ce2e0 0x7f466416a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.015+0000 7f4668eff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f466416d4d0 0x7f4664166850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.015+0000 7f4668eff700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466416a540 con 0x7f4664166d90 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.015+0000 7f4668eff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46640ce6b0 con 0x7f466416d4d0 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.015+0000 7f4668eff700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46640cc8c0 con 0x7f46640ce2e0 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.016+0000 7f466259c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f46640ce2e0 0x7f466416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.016+0000 7f4661d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f466416d4d0 0x7f4664166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.016+0000 7f4661d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f466416d4d0 0x7f4664166850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60898/0 (socket says 172.21.15.89:60898) 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.016+0000 7f4661d9b700 1 -- 172.21.15.89:0/418436418 learned_addr learned my addr 172.21.15.89:0/418436418 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.017+0000 7f4661d9b700 1 -- 172.21.15.89:0/418436418 >> v2:172.21.15.172:3300/0 conn(0x7f46640ce2e0 msgr2=0x7f466416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.017+0000 7f4661d9b700 1 --2- 172.21.15.89:0/418436418 >> v2:172.21.15.172:3300/0 conn(0x7f46640ce2e0 0x7f466416a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.017+0000 7f4661d9b700 1 -- 172.21.15.89:0/418436418 >> v1:172.21.15.89:6789/0 conn(0x7f4664166d90 legacy=0x7f4664070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.017+0000 7f4661d9b700 1 -- 172.21.15.89:0/418436418 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466416ccd0 con 0x7f466416d4d0 2023-07-06T02:23:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.017+0000 7f4661d9b700 1 --2- 172.21.15.89:0/418436418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f466416d4d0 0x7f4664166850 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46640cc3f0 tx=0x7f4654009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923cdee1eba874e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.018+0000 7f466159a700 1 -- 172.21.15.89:0/418436418 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654004230 con 0x7f466416d4d0 2023-07-06T02:23:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.018+0000 7f466159a700 1 -- 172.21.15.89:0/418436418 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4654004390 con 0x7f466416d4d0 2023-07-06T02:23:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.018+0000 7f466159a700 1 -- 172.21.15.89:0/418436418 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540036a0 con 0x7f466416d4d0 2023-07-06T02:23:14.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.019+0000 7f4668eff700 1 -- 172.21.15.89:0/418436418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f466416d4d0 msgr2=0x7f4664166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.019+0000 7f4668eff700 1 --2- 172.21.15.89:0/418436418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f466416d4d0 0x7f4664166850 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46640cc3f0 tx=0x7f4654009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.022+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 --> [v2:172.21.15.89:6800/990084407,v1:172.21.15.89:6801/990084407] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5340ce980 con 0x7fd5200748b0 2023-07-06T02:23:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.022+0000 7fd5317fa700 1 -- 172.21.15.89:0/3574279936 <== osd.0 v2:172.21.15.89:6800/990084407 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd52c003b40 con 0x7fd5200748b0 2023-07-06T02:23:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.025+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 --> [v2:172.21.15.89:6808/3689514906,v1:172.21.15.89:6809/3689514906] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee24002d40 con 0x7fee20074a80 2023-07-06T02:23:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6800/990084407,v1:172.21.15.89:6801/990084407] conn(0x7fd5200748b0 msgr2=0x7fd520076ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fd53ae38700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6800/990084407,v1:172.21.15.89:6801/990084407] conn(0x7fd5200748b0 0x7fd520076ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd52006ace0 msgr2=0x7fd52006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fd53ae38700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd52006ace0 0x7fd52006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd524007fd0 tx=0x7fd524007920 comp rx=0 tx=0).stop 2023-07-06T02:23:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 msgr2=0x7fd53406c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fd53ae38700 1 --2- 172.21.15.89:0/3574279936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5340cd700 0x7fd53406c4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd52800c450 tx=0x7fd52800c810 comp rx=0 tx=0).stop 2023-07-06T02:23:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fd5393d5700 1 -- 172.21.15.89:0/3574279936 reap_dead start 2023-07-06T02:23:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.026+0000 7fee327fc700 1 -- 172.21.15.89:0/2164654905 <== osd.3 v2:172.21.15.89:6808/3689514906 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee24002d40 con 0x7fee20074a80 2023-07-06T02:23:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.027+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 --> [v2:172.21.15.89:6816/3305610859,v1:172.21.15.89:6817/3305610859] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ed8002d40 con 0x7f4ed4074a00 2023-07-06T02:23:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.028+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 shutdown_connections 2023-07-06T02:23:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.028+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 >> 172.21.15.89:0/3574279936 conn(0x7fd5340cc490 msgr2=0x7fd53415c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.028+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 shutdown_connections 2023-07-06T02:23:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.028+0000 7fd53ae38700 1 -- 172.21.15.89:0/3574279936 wait complete. 2023-07-06T02:23:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.028+0000 7f4ee2ffd700 1 -- 172.21.15.89:0/167529713 <== osd.1 v2:172.21.15.89:6816/3305610859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ed8002d40 con 0x7f4ed4074a00 2023-07-06T02:23:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.028+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6816/3305610859,v1:172.21.15.89:6817/3305610859] conn(0x7f4ed4074a00 msgr2=0x7f4ed4076e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.028+0000 7f4ef8999700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6816/3305610859,v1:172.21.15.89:6817/3305610859] conn(0x7f4ed4074a00 0x7f4ed4076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.029+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ed406adb0 msgr2=0x7f4ed406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.029+0000 7f4ef8999700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ed406adb0 0x7f4ed406d260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ca8c0 tx=0x7f4ee8008040 comp rx=0 tx=0).stop 2023-07-06T02:23:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.029+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 msgr2=0x7f4ef406eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.029+0000 7f4ef8999700 1 --2- 172.21.15.89:0/167529713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef40b58d0 0x7f4ef406eb60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40afe90 tx=0x7f4ee4004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.029+0000 7f4ef2d9d700 1 -- 172.21.15.89:0/167529713 reap_dead start 2023-07-06T02:23:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.029+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 shutdown_connections 2023-07-06T02:23:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.029+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 >> 172.21.15.89:0/167529713 conn(0x7f4ef40d0b00 msgr2=0x7f4ef4163270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.033+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 --> [v2:172.21.15.172:6820/466302387,v1:172.21.15.172:6822/466302387] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb25c0cdb20 con 0x7fb25c150da0 2023-07-06T02:23:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 shutdown_connections 2023-07-06T02:23:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7f4ef8999700 1 -- 172.21.15.89:0/167529713 wait complete. 2023-07-06T02:23:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6808/3689514906,v1:172.21.15.89:6809/3689514906] conn(0x7fee20074a80 msgr2=0x7fee20076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6808/3689514906,v1:172.21.15.89:6809/3689514906] conn(0x7fee20074a80 0x7fee20076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fb24affd700 1 -- 172.21.15.89:0/3298223525 <== osd.6 v2:172.21.15.172:6820/466302387 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb254009b10 con 0x7fb25c150da0 2023-07-06T02:23:14.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fee2006adb0 msgr2=0x7fee2006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fee2006adb0 0x7fee2006d260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0c9e10 tx=0x7fee34001c50 comp rx=0 tx=0).stop 2023-07-06T02:23:14.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee3c0cd910 msgr2=0x7fee3c0c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 --2- 172.21.15.89:0/2164654905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee3c0cd910 0x7fee3c0c59f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c450 tx=0x7fee3800c760 comp rx=0 tx=0).stop 2023-07-06T02:23:14.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee4228f700 1 -- 172.21.15.89:0/2164654905 reap_dead start 2023-07-06T02:23:14.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 shutdown_connections 2023-07-06T02:23:14.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.034+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 >> 172.21.15.89:0/2164654905 conn(0x7fee3c0cf1c0 msgr2=0x7fee3c0affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7f4668eff700 1 -- 172.21.15.89:0/418436418 shutdown_connections 2023-07-06T02:23:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7f4668eff700 1 --2- 172.21.15.89:0/418436418 >> v2:172.21.15.172:3300/0 conn(0x7f46640ce2e0 0x7f466416a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7f4668eff700 1 --2- 172.21.15.89:0/418436418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f466416d4d0 0x7f4664166850 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7f4668eff700 1 -- 172.21.15.89:0/418436418 >> 172.21.15.89:0/418436418 conn(0x7f46640ce870 msgr2=0x7f46640cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7f4668eff700 1 -- 172.21.15.89:0/418436418 shutdown_connections 2023-07-06T02:23:14.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7f4668eff700 1 -- 172.21.15.89:0/418436418 wait complete. 2023-07-06T02:23:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 shutdown_connections 2023-07-06T02:23:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.036+0000 7fee43cf2700 1 -- 172.21.15.89:0/2164654905 wait complete. 2023-07-06T02:23:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.037+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:6820/466302387,v1:172.21.15.172:6822/466302387] conn(0x7fb25c150da0 msgr2=0x7fb25c06db70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.037+0000 7fb2609ca700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:6820/466302387,v1:172.21.15.172:6822/466302387] conn(0x7fb25c150da0 0x7fb25c06db70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.038+0000 7f4668eff700 1 Processor -- start 2023-07-06T02:23:14.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.038+0000 7f4668eff700 1 -- start start 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4668eff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46640ce2e0 0x7f46640b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4668eff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664166d90 0x7f46641f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4668eff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f466416d4d0 0x7f46641582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4668eff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46640c7040 con 0x7f46640ce2e0 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4668eff700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664158c90 con 0x7f466416d4d0 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4668eff700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46640c69f0 con 0x7f4664166d90 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664166d90 0x7f46641f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664166d90 0x7f46641f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55880/0 (socket says 172.21.15.89:55880) 2023-07-06T02:23:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 -- 172.21.15.89:0/3791155725 learned_addr learned my addr 172.21.15.89:0/3791155725 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4662d9d700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f466416d4d0 0x7f46641582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 -- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f466416d4d0 msgr2=0x7f46641582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f466416d4d0 0x7f46641582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 -- 172.21.15.89:0/3791155725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46640ce2e0 msgr2=0x7f46640b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46640ce2e0 0x7f46640b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 -- 172.21.15.89:0/3791155725 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4654009550 con 0x7f4664166d90 2023-07-06T02:23:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.039+0000 7f4661d9b700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664166d90 0x7f46641f3070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4654004000 tx=0x7f4654004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb23c06ace0 msgr2=0x7fb23c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7fb2609ca700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb23c06ace0 0x7fb23c06d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005910 tx=0x7fb24c009040 comp rx=0 tx=0).stop 2023-07-06T02:23:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb25c1608d0 msgr2=0x7fb25c150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7fb2609ca700 1 --2- 172.21.15.89:0/3298223525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb25c1608d0 0x7fb25c150860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb250016000 tx=0x7fb250004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654003e10 con 0x7f4664166d90 2023-07-06T02:23:14.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46540045d0 con 0x7f4664166d90 2023-07-06T02:23:14.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654011830 con 0x7f4664166d90 2023-07-06T02:23:14.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46640c6cd0 con 0x7f4664166d90 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.040+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46641f35b0 con 0x7f4664166d90 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.041+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46640ccba0 con 0x7f4664166d90 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.042+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4654022020 con 0x7f4664166d90 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.042+0000 7f4652ffd700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f464406acb0 0x7f464406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.042+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f465408ad20 con 0x7f4664166d90 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.042+0000 7f4652ffd700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:6816/3226277044,v1:172.21.15.172:6817/3226277044] conn(0x7f4644074900 0x7f4644076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.042+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 --> [v2:172.21.15.172:6816/3226277044,v1:172.21.15.172:6817/3226277044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4644077400 con 0x7f4644074900 2023-07-06T02:23:14.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.042+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f465408d3e0 con 0x7f4664166d90 2023-07-06T02:23:14.099 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T02:23:14.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.044+0000 7f4662d9d700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:6816/3226277044,v1:172.21.15.172:6817/3226277044] conn(0x7f4644074900 0x7f4644076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.044+0000 7f4662d9d700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:6816/3226277044,v1:172.21.15.172:6817/3226277044] conn(0x7f4644074900 0x7f4644076d30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.045+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== osd.7 v2:172.21.15.172:6816/3226277044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4644077400 con 0x7f4644074900 2023-07-06T02:23:14.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.046+0000 7fb25ad9d700 1 -- 172.21.15.89:0/3298223525 reap_dead start 2023-07-06T02:23:14.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.046+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 shutdown_connections 2023-07-06T02:23:14.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.046+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 >> 172.21.15.89:0/3298223525 conn(0x7fb25c0cf8d0 msgr2=0x7fb25c06cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.046+0000 7f466259c700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f464406acb0 0x7f464406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.046+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 shutdown_connections 2023-07-06T02:23:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.046+0000 7fb2609ca700 1 -- 172.21.15.89:0/3298223525 wait complete. 2023-07-06T02:23:14.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.046+0000 7f466259c700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f464406acb0 0x7f464406d160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f465800ffd0 tx=0x7f465800f2c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.048+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 --> [v2:172.21.15.172:6800/1682295071,v1:172.21.15.172:6801/1682295071] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec100d21c0 con 0x7febf4074900 2023-07-06T02:23:14.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.048+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 --> [v2:172.21.15.172:6808/2248870656,v1:172.21.15.172:6809/2248870656] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2e400ce6a0 con 0x7f2e2c074a00 2023-07-06T02:23:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.048+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 --> [v2:172.21.15.89:6820/1124333878,v1:172.21.15.89:6822/1124333878] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5988005ca0 con 0x7f59880015e0 2023-07-06T02:23:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.049+0000 7f59967fc700 1 -- 172.21.15.89:0/1855358744 <== osd.2 v2:172.21.15.89:6820/1124333878 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5988005ca0 con 0x7f59880015e0 2023-07-06T02:23:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.049+0000 7fec067fc700 1 -- 172.21.15.89:0/1821441900 <== osd.4 v2:172.21.15.172:6800/1682295071 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec08003e50 con 0x7febf4074900 2023-07-06T02:23:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.049+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6820/1124333878,v1:172.21.15.89:6822/1124333878] conn(0x7f59880015e0 msgr2=0x7f5988003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.049+0000 7f59a7f94700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6820/1124333878,v1:172.21.15.89:6822/1124333878] conn(0x7f59880015e0 0x7f5988003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f598406ad30 msgr2=0x7f598406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7f59a7f94700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f598406ad30 0x7f598406d1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f599c0090b0 tx=0x7f599c01c040 comp rx=0 tx=0).stop 2023-07-06T02:23:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59a00c8170 msgr2=0x7f59a0201a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7f59a7f94700 1 --2- 172.21.15.89:0/1855358744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59a00c8170 0x7f59a0201a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f599800cca0 tx=0x7f599800a060 comp rx=0 tx=0).stop 2023-07-06T02:23:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7f59a6531700 1 -- 172.21.15.89:0/1855358744 reap_dead start 2023-07-06T02:23:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 shutdown_connections 2023-07-06T02:23:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 >> 172.21.15.89:0/1855358744 conn(0x7f59a00cc490 msgr2=0x7f59a00cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:6800/1682295071,v1:172.21.15.172:6801/1682295071] conn(0x7febf4074900 msgr2=0x7febf4076d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.172:6800/1682295071,v1:172.21.15.172:6801/1682295071] conn(0x7febf4074900 0x7febf4076d30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febf406acb0 msgr2=0x7febf406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febf406acb0 0x7febf406d160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec00006fd0 tx=0x7fec00009190 comp rx=0 tx=0).stop 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 msgr2=0x7fec101f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 --2- 172.21.15.89:0/1821441900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec100ceda0 0x7fec101f32c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00b450 tx=0x7fec0c00b760 comp rx=0 tx=0).stop 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec16674700 1 -- 172.21.15.89:0/1821441900 reap_dead start 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 shutdown_connections 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 >> 172.21.15.89:0/1821441900 conn(0x7fec100b5390 msgr2=0x7fec100b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 shutdown_connections 2023-07-06T02:23:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.050+0000 7fec180d7700 1 -- 172.21.15.89:0/1821441900 wait complete. 2023-07-06T02:23:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.051+0000 7f2e27fff700 1 -- 172.21.15.89:0/3805513073 <== osd.5 v2:172.21.15.172:6808/2248870656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2e34009460 con 0x7f2e2c074a00 2023-07-06T02:23:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.052+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:6808/2248870656,v1:172.21.15.172:6809/2248870656] conn(0x7f2e2c074a00 msgr2=0x7f2e2c076e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.052+0000 7f2e45a46700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:6808/2248870656,v1:172.21.15.172:6809/2248870656] conn(0x7f2e2c074a00 0x7f2e2c076e30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.052+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2e2c06ad30 msgr2=0x7f2e2c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.052+0000 7f2e45a46700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f2e2c06ad30 0x7f2e2c06d1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800f300 tx=0x7f2e380042c0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.052+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e400cd700 msgr2=0x7f2e400c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.052+0000 7f2e45a46700 1 --2- 172.21.15.89:0/3805513073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e400cd700 0x7f2e400c6f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e4006f2b0 tx=0x7f2e30004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.052+0000 7f2e3f7fe700 1 -- 172.21.15.89:0/3805513073 reap_dead start 2023-07-06T02:23:14.111 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.111 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T02:23:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.056+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 shutdown_connections 2023-07-06T02:23:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.056+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 >> 172.21.15.89:0/3805513073 conn(0x7f2e400cc490 msgr2=0x7f2e400cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.056+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 shutdown_connections 2023-07-06T02:23:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.056+0000 7f2e45a46700 1 -- 172.21.15.89:0/3805513073 wait complete. 2023-07-06T02:23:14.113 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-07-06T02:23:14.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T02:23:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.060+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 shutdown_connections 2023-07-06T02:23:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.063+0000 7f59a7f94700 1 -- 172.21.15.89:0/1855358744 wait complete. 2023-07-06T02:23:14.115 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.115 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T02:23:14.116 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-07-06T02:23:14.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T02:23:14.117 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-07-06T02:23:14.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T02:23:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.071+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 --> [v2:172.21.15.172:6816/3226277044,v1:172.21.15.172:6817/3226277044] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f466416cfb0 con 0x7f4644074900 2023-07-06T02:23:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4652ffd700 1 -- 172.21.15.89:0/3791155725 <== osd.7 v2:172.21.15.172:6816/3226277044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f465c001cc0 con 0x7f4644074900 2023-07-06T02:23:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:6816/3226277044,v1:172.21.15.172:6817/3226277044] conn(0x7f4644074900 msgr2=0x7f4644076d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:6816/3226277044,v1:172.21.15.172:6817/3226277044] conn(0x7f4644074900 0x7f4644076d30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f464406acb0 msgr2=0x7f464406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f464406acb0 0x7f464406d160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f465800ffd0 tx=0x7f465800f2c0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664166d90 msgr2=0x7f46641f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 --2- 172.21.15.89:0/3791155725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664166d90 0x7f46641f3070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4654004000 tx=0x7f4654004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4662d9d700 1 -- 172.21.15.89:0/3791155725 reap_dead start 2023-07-06T02:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 shutdown_connections 2023-07-06T02:23:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.072+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 >> 172.21.15.89:0/3791155725 conn(0x7f46640ce870 msgr2=0x7f4664070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.073+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 shutdown_connections 2023-07-06T02:23:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.073+0000 7f4668eff700 1 -- 172.21.15.89:0/3791155725 wait complete. 2023-07-06T02:23:14.121 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T02:23:14.122 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.123 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T02:23:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.359+0000 7fd096f54700 1 Processor -- start 2023-07-06T02:23:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.359+0000 7f629f46b700 1 Processor -- start 2023-07-06T02:23:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.358+0000 7f21037aa700 1 Processor -- start 2023-07-06T02:23:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.358+0000 7f21037aa700 1 -- start start 2023-07-06T02:23:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.359+0000 7f21037aa700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f20fc070580 0x7f20fc070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.359+0000 7f21037aa700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc070e90 con 0x7f20fc166ac0 2023-07-06T02:23:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7f21037aa700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0cc5e0 con 0x7f20fc0ce2e0 2023-07-06T02:23:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7f21037aa700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc0cc8c0 con 0x7f20fc070580 2023-07-06T02:23:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7f629f46b700 1 -- start start 2023-07-06T02:23:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f629f46b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62980707f0 0x7f629806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f629f46b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6298141870 0x7f629806cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f629f46b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629806a960 con 0x7f6298070bc0 2023-07-06T02:23:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f629f46b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62980cd100 con 0x7f6298141870 2023-07-06T02:23:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f629f46b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629806ac40 con 0x7f62980707f0 2023-07-06T02:23:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.359+0000 7fd096f54700 1 -- start start 2023-07-06T02:23:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fd096f54700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd090163e60 0x7fd090070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fd096f54700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd09006a060 0x7fd0900cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fd096f54700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09006dd70 con 0x7fd0900ad750 2023-07-06T02:23:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fd096f54700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd090164230 con 0x7fd090163e60 2023-07-06T02:23:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fd096f54700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09006ba10 con 0x7fd09006a060 2023-07-06T02:23:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fd08ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd09006a060 0x7fd0900cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd09006a060 0x7fd0900cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59676/0 (socket says 172.21.15.89:59676) 2023-07-06T02:23:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08ffff700 1 -- 172.21.15.89:0/3524092330 learned_addr learned my addr 172.21.15.89:0/3524092330 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08ffff700 1 -- 172.21.15.89:0/3524092330 >> v2:172.21.15.89:3300/0 conn(0x7fd090163e60 msgr2=0x7fd090070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08ffff700 1 --2- 172.21.15.89:0/3524092330 >> v2:172.21.15.89:3300/0 conn(0x7fd090163e60 0x7fd090070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08ffff700 1 -- 172.21.15.89:0/3524092330 >> v1:172.21.15.89:6789/0 conn(0x7fd0900ad750 legacy=0x7fd0901643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08ffff700 1 -- 172.21.15.89:0/3524092330 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0900ce470 con 0x7fd09006a060 2023-07-06T02:23:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fa3857db700 1 Processor -- start 2023-07-06T02:23:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7fa3857db700 1 -- start start 2023-07-06T02:23:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08ffff700 1 --2- 172.21.15.89:0/3524092330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 0x7fd0900cdf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd0900cc3f0 tx=0x7fd080009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb38cc0ecf20d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f629ca06700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6298141870 0x7f629806cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f629ca06700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6298141870 0x7f629806cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60928/0 (socket says 172.21.15.89:60928) 2023-07-06T02:23:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7f629ca06700 1 -- 172.21.15.89:0/2934020702 learned_addr learned my addr 172.21.15.89:0/2934020702 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f2100d45700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f20fc0ce2e0 0x7f20fc16d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33342/0 (socket says 172.21.15.89:33342) 2023-07-06T02:23:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7f2100d45700 1 -- 172.21.15.89:0/2896853922 learned_addr learned my addr 172.21.15.89:0/2896853922 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.360+0000 7f2101546700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f20fc166ac0 0x7f20fc16a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57582/0 (socket says 172.21.15.89:57582) 2023-07-06T02:23:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fd08f7fe700 1 -- 172.21.15.89:0/3524092330 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000b870 con 0x7fd09006a060 2023-07-06T02:23:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fd08f7fe700 1 -- 172.21.15.89:0/3524092330 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08000b9d0 con 0x7fd09006a060 2023-07-06T02:23:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fd08f7fe700 1 -- 172.21.15.89:0/3524092330 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000bd60 con 0x7fd09006a060 2023-07-06T02:23:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fd096f54700 1 -- 172.21.15.89:0/3524092330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 msgr2=0x7fd0900cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fd096f54700 1 --2- 172.21.15.89:0/3524092330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 0x7fd0900cdf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd0900cc3f0 tx=0x7fd080009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f629d207700 1 --2- 172.21.15.89:0/2934020702 >> v2:172.21.15.172:3300/0 conn(0x7f62980707f0 0x7f629806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f629ca06700 1 -- 172.21.15.89:0/2934020702 >> v2:172.21.15.172:3300/0 conn(0x7f62980707f0 msgr2=0x7f629806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f629ca06700 1 --2- 172.21.15.89:0/2934020702 >> v2:172.21.15.172:3300/0 conn(0x7f62980707f0 0x7f629806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f629ca06700 1 -- 172.21.15.89:0/2934020702 >> v1:172.21.15.172:6789/0 conn(0x7f6298070bc0 legacy=0x7f629806d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f629ca06700 1 -- 172.21.15.89:0/2934020702 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629806af20 con 0x7f6298141870 2023-07-06T02:23:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fd096f54700 1 -- 172.21.15.89:0/3524092330 shutdown_connections 2023-07-06T02:23:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fd096f54700 1 --2- 172.21.15.89:0/3524092330 >> v2:172.21.15.89:3300/0 conn(0x7fd090163e60 0x7fd090070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fd096f54700 1 --2- 172.21.15.89:0/3524092330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 0x7fd0900cdf30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fd096f54700 1 -- 172.21.15.89:0/3524092330 >> 172.21.15.89:0/3524092330 conn(0x7fd0900ce870 msgr2=0x7fd0900cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f629ca06700 1 --2- 172.21.15.89:0/2934020702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298141870 0x7f629806cca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f62980ce7d0 tx=0x7f628800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2456b7b66113e52e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f2101d47700 1 --2- 172.21.15.89:0/2896853922 >> v2:172.21.15.172:3300/0 conn(0x7f20fc070580 0x7f20fc070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582646047 0 0) 0x7f20fc0cc5e0 con 0x7f20fc0ce2e0 2023-07-06T02:23:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e80035b0 con 0x7f20fc0ce2e0 2023-07-06T02:23:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951206484 0 0) 0x7f20fc070e90 con 0x7f20fc166ac0 2023-07-06T02:23:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20fc0cc5e0 con 0x7f20fc166ac0 2023-07-06T02:23:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157040103 0 0) 0x7f20fc0cc5e0 con 0x7f20fc166ac0 2023-07-06T02:23:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 >> v2:172.21.15.172:3300/0 conn(0x7f20fc070580 msgr2=0x7f20fc070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 --2- 172.21.15.89:0/2896853922 >> v2:172.21.15.172:3300/0 conn(0x7f20fc070580 0x7f20fc070950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 >> v1:172.21.15.172:6789/0 conn(0x7f20fc0ce2e0 legacy=0x7f20fc16d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc16ccd0 con 0x7f20fc166ac0 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f20ec001670 con 0x7f20fc166ac0 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 >> v1:172.21.15.89:6789/0 conn(0x7f20fc166ac0 legacy=0x7f20fc16a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f20f3fff700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e8004bf0 0x7f20e8004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f20f3fff700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e80037f0 0x7f20e8003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f20f3fff700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20e80042c0 0x7f20e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0cc8c0 con 0x7f20e80037f0 2023-07-06T02:23:14.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e800c1e0 con 0x7f20e80042c0 2023-07-06T02:23:14.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e800c3a0 con 0x7f20e8004bf0 2023-07-06T02:23:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fd096f54700 1 -- 172.21.15.89:0/3524092330 shutdown_connections 2023-07-06T02:23:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fa3857db700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38016a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fa3857db700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa380070580 0x7fa380070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fa3857db700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa380070ec0 con 0x7fa38016d4d0 2023-07-06T02:23:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fa3857db700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3800ce6b0 con 0x7fa380070580 2023-07-06T02:23:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.361+0000 7fa3857db700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38016a540 con 0x7fa3800ce2e0 2023-07-06T02:23:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37f7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa380070580 0x7fa380070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37e7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa38016d4d0 0x7fa380166850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57602/0 (socket says 172.21.15.89:57602) 2023-07-06T02:23:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37e7fc700 1 -- 172.21.15.89:0/1094265106 learned_addr learned my addr 172.21.15.89:0/1094265106 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37effd700 1 --2- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37dffb700 1 -- 172.21.15.89:0/1094265106 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124662237 0 0) 0x7fa380070ec0 con 0x7fa38016d4d0 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37dffb700 1 -- 172.21.15.89:0/1094265106 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3680145c0 con 0x7fa38016d4d0 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37f7fe700 1 -- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 msgr2=0x7fa38016a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37f7fe700 1 --2- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38016a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37f7fe700 1 -- 172.21.15.89:0/1094265106 >> v1:172.21.15.89:6789/0 conn(0x7fa38016d4d0 legacy=0x7fa380166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.362+0000 7fa37f7fe700 1 -- 172.21.15.89:0/1094265106 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800cc8c0 con 0x7fa380070580 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fa37f7fe700 1 --2- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380070580 0x7fa380070980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa370014450 tx=0x7fa370014760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9bab4ee3a922abd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fa37dffb700 1 -- 172.21.15.89:0/1094265106 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370005e30 con 0x7fa380070580 2023-07-06T02:23:14.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fa37dffb700 1 -- 172.21.15.89:0/1094265106 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37000f0d0 con 0x7fa380070580 2023-07-06T02:23:14.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.363+0000 7fa37dffb700 1 -- 172.21.15.89:0/1094265106 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370015640 con 0x7fa380070580 2023-07-06T02:23:14.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f628ffff700 1 -- 172.21.15.89:0/2934020702 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288010070 con 0x7f6298141870 2023-07-06T02:23:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f628ffff700 1 -- 172.21.15.89:0/2934020702 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800d990 con 0x7f6298141870 2023-07-06T02:23:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f628ffff700 1 -- 172.21.15.89:0/2934020702 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800dd00 con 0x7f6298141870 2023-07-06T02:23:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f2101546700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e8004bf0 0x7f20e8004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fd096f54700 1 -- 172.21.15.89:0/3524092330 wait complete. 2023-07-06T02:23:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f2100d45700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e80037f0 0x7f20e8003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f2101546700 1 -- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20e80042c0 msgr2=0x7f20e800ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f2101546700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20e80042c0 0x7f20e800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f2101546700 1 -- 172.21.15.89:0/2896853922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e80037f0 msgr2=0x7f20e8003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f2101546700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20e80037f0 0x7f20e8003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7f2101546700 1 -- 172.21.15.89:0/2896853922 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc16ccd0 con 0x7f20e8004bf0 2023-07-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f2101546700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e8004bf0 0x7f20e8004fe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00c1b0 tx=0x7f20e8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd1ed3bb7171531 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f2101d47700 1 -- 172.21.15.89:0/2896853922 reap_dead start 2023-07-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 -- 172.21.15.89:0/2934020702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298141870 msgr2=0x7f629806cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 --2- 172.21.15.89:0/2934020702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298141870 0x7f629806cca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f62980ce7d0 tx=0x7f628800d330 comp rx=0 tx=0).stop 2023-07-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 -- 172.21.15.89:0/2934020702 shutdown_connections 2023-07-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 --2- 172.21.15.89:0/2934020702 >> v2:172.21.15.172:3300/0 conn(0x7f62980707f0 0x7f629806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 --2- 172.21.15.89:0/2934020702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298141870 0x7f629806cca0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 -- 172.21.15.89:0/2934020702 >> 172.21.15.89:0/2934020702 conn(0x7f62980ce870 msgr2=0x7f62980cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 -- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380070580 msgr2=0x7fa380070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 --2- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380070580 0x7fa380070980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa370014450 tx=0x7fa370014760 comp rx=0 tx=0).stop 2023-07-06T02:23:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 -- 172.21.15.89:0/1094265106 shutdown_connections 2023-07-06T02:23:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 --2- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa380070580 0x7fa380070980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 --2- 172.21.15.89:0/1094265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38016a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 -- 172.21.15.89:0/1094265106 >> 172.21.15.89:0/1094265106 conn(0x7fa3800ce870 msgr2=0x7fa3800cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 -- 172.21.15.89:0/1094265106 shutdown_connections 2023-07-06T02:23:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.364+0000 7fa3857db700 1 -- 172.21.15.89:0/1094265106 wait complete. 2023-07-06T02:23:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 -- 172.21.15.89:0/2934020702 shutdown_connections 2023-07-06T02:23:14.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.366+0000 7fd096f54700 1 Processor -- start 2023-07-06T02:23:14.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.366+0000 7fa3857db700 1 Processor -- start 2023-07-06T02:23:14.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.366+0000 7fa3857db700 1 -- start start 2023-07-06T02:23:14.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.366+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00ccd0 con 0x7f20e8004bf0 2023-07-06T02:23:14.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.366+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec00ce50 con 0x7f20e8004bf0 2023-07-06T02:23:14.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.366+0000 7f20f3fff700 1 -- 172.21.15.89:0/2896853922 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec005ae0 con 0x7f20e8004bf0 2023-07-06T02:23:14.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7f21037aa700 1 -- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e8004bf0 msgr2=0x7f20e8004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7f21037aa700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e8004bf0 0x7f20e8004fe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00c1b0 tx=0x7f20e8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:14.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7f21037aa700 1 -- 172.21.15.89:0/2896853922 shutdown_connections 2023-07-06T02:23:14.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7f21037aa700 1 --2- 172.21.15.89:0/2896853922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e8004bf0 0x7f20e8004fe0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00c1b0 tx=0x7f20e8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:14.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7f21037aa700 1 -- 172.21.15.89:0/2896853922 >> 172.21.15.89:0/2896853922 conn(0x7f20fc0ce870 msgr2=0x7f20fc0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.365+0000 7f629f46b700 1 -- 172.21.15.89:0/2934020702 wait complete. 2023-07-06T02:23:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7f629f46b700 1 Processor -- start 2023-07-06T02:23:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3857db700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa380070580 0x7fa38007f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3857db700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38007f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3857db700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38016d4d0 0x7fa3800c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3857db700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800878d0 con 0x7fa380070580 2023-07-06T02:23:14.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3857db700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa380087bb0 con 0x7fa38016d4d0 2023-07-06T02:23:14.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3857db700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa380087e90 con 0x7fa3800ce2e0 2023-07-06T02:23:14.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38016d4d0 0x7fa3800c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37e7fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38007f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38016d4d0 0x7fa3800c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59706/0 (socket says 172.21.15.89:59706) 2023-07-06T02:23:14.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa380070580 0x7fa38007f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37f7fe700 1 -- 172.21.15.89:0/971264450 learned_addr learned my addr 172.21.15.89:0/971264450 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37effd700 1 -- 172.21.15.89:0/971264450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 msgr2=0x7fa38007f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37effd700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38007f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37effd700 1 -- 172.21.15.89:0/971264450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38016d4d0 msgr2=0x7fa3800c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37effd700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38016d4d0 0x7fa3800c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37effd700 1 -- 172.21.15.89:0/971264450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37000fcd0 con 0x7fa380070580 2023-07-06T02:23:14.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fa37effd700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa380070580 0x7fa38007f370 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa378007e60 tx=0x7fa37800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7f629f46b700 1 -- start start 2023-07-06T02:23:14.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629f46b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62980707f0 0x7f62981e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629f46b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298070bc0 0x7f6298202ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629f46b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62980c0f10 0x7f62980c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629f46b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62981e1880 con 0x7f6298070bc0 2023-07-06T02:23:14.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629f46b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c5940 con 0x7f62980707f0 2023-07-06T02:23:14.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629f46b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c6150 con 0x7f62980c0f10 2023-07-06T02:23:14.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629ca06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298070bc0 0x7f6298202ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629ca06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298070bc0 0x7f6298202ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60956/0 (socket says 172.21.15.89:60956) 2023-07-06T02:23:14.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f629ca06700 1 -- 172.21.15.89:0/2851805971 learned_addr learned my addr 172.21.15.89:0/2851805971 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629d207700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62980707f0 0x7f62981e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7f21037aa700 1 -- 172.21.15.89:0/2896853922 shutdown_connections 2023-07-06T02:23:14.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3bfee7700 1 Processor -- start 2023-07-06T02:23:14.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fa3bfee7700 1 -- start start 2023-07-06T02:23:14.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fa3bfee7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3b806dcb0 0x7fa3b815c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fa3bfee7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b806a960 con 0x7fa3b80707f0 2023-07-06T02:23:14.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fa3bfee7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b806ac40 con 0x7fa3b8141870 2023-07-06T02:23:14.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fa3bfee7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b806a770 con 0x7fa3b806dcb0 2023-07-06T02:23:14.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.366+0000 7fd096f54700 1 -- start start 2023-07-06T02:23:14.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd096f54700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 0x7fd09015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd096f54700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900ad750 0x7fd0900b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd096f54700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd090163e60 0x7fd0901493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd096f54700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09015ca20 con 0x7fd0900ad750 2023-07-06T02:23:14.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd096f54700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09015ba90 con 0x7fd09006a060 2023-07-06T02:23:14.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd096f54700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd090149f50 con 0x7fd090163e60 2023-07-06T02:23:14.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900ad750 0x7fd0900b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900ad750 0x7fd0900b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60942/0 (socket says 172.21.15.89:60942) 2023-07-06T02:23:14.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.367+0000 7fd08ffff700 1 -- 172.21.15.89:0/4077302946 learned_addr learned my addr 172.21.15.89:0/4077302946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fd08ffff700 1 -- 172.21.15.89:0/4077302946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd090163e60 msgr2=0x7fd0901493e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fd08ffff700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd090163e60 0x7fd0901493e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fd08ffff700 1 -- 172.21.15.89:0/4077302946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 msgr2=0x7fd09015c4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fd08ffff700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 0x7fd09015c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fd08ffff700 1 -- 172.21.15.89:0/4077302946 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd080009550 con 0x7fd0900ad750 2023-07-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fd08ffff700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900ad750 0x7fd0900b3e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd09006d7a0 tx=0x7fd080004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.368+0000 7fd08d7fa700 1 -- 172.21.15.89:0/4077302946 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08001b980 con 0x7fd0900ad750 2023-07-06T02:23:14.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fd08d7fa700 1 -- 172.21.15.89:0/4077302946 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08001bb00 con 0x7fd0900ad750 2023-07-06T02:23:14.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fd08d7fa700 1 -- 172.21.15.89:0/4077302946 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080011420 con 0x7fd0900ad750 2023-07-06T02:23:14.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09014a230 con 0x7fd0900ad750 2023-07-06T02:23:14.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09015bfc0 con 0x7fd0900ad750 2023-07-06T02:23:14.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fa3be484700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3b806dcb0 0x7fa3b815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629da08700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62980c0f10 0x7f62980c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629ca06700 1 -- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62980c0f10 msgr2=0x7f62980c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629ca06700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62980c0f10 0x7f62980c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629ca06700 1 -- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62980707f0 msgr2=0x7f62981e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629ca06700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62980707f0 0x7f62981e3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629ca06700 1 -- 172.21.15.89:0/2851805971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62880085d0 con 0x7f6298070bc0 2023-07-06T02:23:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629da08700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62980c0f10 0x7f62980c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fa3bd482700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa3b8141870 0x7fa3b815b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33388/0 (socket says 172.21.15.89:33388) 2023-07-06T02:23:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7fa3bd482700 1 -- 172.21.15.89:0/2163222150 learned_addr learned my addr 172.21.15.89:0/2163222150 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f629ca06700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298070bc0 0x7f6298202ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6288000c00 tx=0x7f6288004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.369+0000 7f21037aa700 1 -- 172.21.15.89:0/2896853922 wait complete. 2023-07-06T02:23:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7faba899e700 1 Processor -- start 2023-07-06T02:23:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7faba899e700 1 -- start start 2023-07-06T02:23:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574177833 0 0) 0x7fa3b806ac40 con 0x7fa3b8141870 2023-07-06T02:23:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a40034d0 con 0x7fa3b8141870 2023-07-06T02:23:14.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7fa3677fe700 1 -- 172.21.15.89:0/971264450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800ede0 con 0x7fa380070580 2023-07-06T02:23:14.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7fa3677fe700 1 -- 172.21.15.89:0/971264450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378014420 con 0x7fa380070580 2023-07-06T02:23:14.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7fa3677fe700 1 -- 172.21.15.89:0/971264450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3780147b0 con 0x7fa380070580 2023-07-06T02:23:14.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7fceabf11700 1 Processor -- start 2023-07-06T02:23:14.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7fceabf11700 1 -- start start 2023-07-06T02:23:14.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7f21037aa700 1 Processor -- start 2023-07-06T02:23:14.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa380088170 con 0x7fa380070580 2023-07-06T02:23:14.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa380088730 con 0x7fa380070580 2023-07-06T02:23:14.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7f21037aa700 1 -- start start 2023-07-06T02:23:14.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7faba899e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faba41610d0 0x7faba40cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7faba899e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40b8850 0x7faba406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7faba899e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba41646c0 con 0x7faba40cd970 2023-07-06T02:23:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7faba899e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba415b620 con 0x7faba41610d0 2023-07-06T02:23:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7faba899e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba40d1160 con 0x7faba40b8850 2023-07-06T02:23:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba259c700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7faba40cd970 0x7faba40d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33394/0 (socket says 172.21.15.89:33394) 2023-07-06T02:23:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba259c700 1 -- 172.21.15.89:0/727148001 learned_addr learned my addr 172.21.15.89:0/727148001 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba2d9d700 1 --2- 172.21.15.89:0/727148001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40b8850 0x7faba406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba159a700 1 -- 172.21.15.89:0/727148001 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294199485 0 0) 0x7faba41646c0 con 0x7faba40cd970 2023-07-06T02:23:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba159a700 1 -- 172.21.15.89:0/727148001 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab880035b0 con 0x7faba40cd970 2023-07-06T02:23:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba2d9d700 1 -- 172.21.15.89:0/727148001 >> v2:172.21.15.89:3300/0 conn(0x7faba41610d0 msgr2=0x7faba40cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba2d9d700 1 --2- 172.21.15.89:0/727148001 >> v2:172.21.15.89:3300/0 conn(0x7faba41610d0 0x7faba40cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba2d9d700 1 -- 172.21.15.89:0/727148001 >> v1:172.21.15.172:6789/0 conn(0x7faba40cd970 legacy=0x7faba40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba2d9d700 1 -- 172.21.15.89:0/727148001 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4161650 con 0x7faba40b8850 2023-07-06T02:23:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fceabf11700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcea406bed0 0x7fcea406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fceabf11700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea406a6e0 con 0x7fcea406b540 2023-07-06T02:23:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fceabf11700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea406a100 con 0x7fcea4160d60 2023-07-06T02:23:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fceabf11700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40cf0b0 con 0x7fcea406bed0 2023-07-06T02:23:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fceaa4ae700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcea406b540 0x7fcea4164630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57644/0 (socket says 172.21.15.89:57644) 2023-07-06T02:23:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fceaa4ae700 1 -- 172.21.15.89:0/903588143 learned_addr learned my addr 172.21.15.89:0/903588143 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.371+0000 7fd08d7fa700 1 -- 172.21.15.89:0/4077302946 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd080011580 con 0x7fd0900ad750 2023-07-06T02:23:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fd08d7fa700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd07406ad80 0x7fd07406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7fd08d7fa700 1 -- 172.21.15.89:0/4077302946 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd08008ad60 con 0x7fd0900ad750 2023-07-06T02:23:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7faba2d9d700 1 --2- 172.21.15.89:0/727148001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40b8850 0x7faba406bed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d90 tx=0x7fab9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88feb831f1e52c85 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7f21037aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20fc207100 0x7f20fc16d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7f21037aa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc2074d0 0x7f20fc16d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7f21037aa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc16e1f0 0x7f20fc16e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7f21037aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc16eba0 con 0x7f20fc207100 2023-07-06T02:23:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7f21037aa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0c2b00 con 0x7f20fc2074d0 2023-07-06T02:23:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.372+0000 7f21037aa700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0c2de0 con 0x7f20fc16e1f0 2023-07-06T02:23:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc16e1f0 0x7f20fc16e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc16e1f0 0x7f20fc16e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:55956/0 (socket says 172.21.15.89:55956) 2023-07-06T02:23:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 -- 172.21.15.89:0/3759547699 learned_addr learned my addr 172.21.15.89:0/3759547699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7faba159a700 1 -- 172.21.15.89:0/727148001 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00e070 con 0x7faba40b8850 2023-07-06T02:23:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fd094cf0700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd07406ad80 0x7fd07406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2954638422 0 0) 0x7fa3a40034d0 con 0x7fa3b8141870 2023-07-06T02:23:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3b806dcb0 msgr2=0x7fa3b815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3b806dcb0 0x7fa3b815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 >> v1:172.21.15.89:6789/0 conn(0x7fa3b80707f0 legacy=0x7fa3b806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b806af20 con 0x7fa3b8141870 2023-07-06T02:23:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fa3a80016b0 con 0x7fa3b8141870 2023-07-06T02:23:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7faba159a700 1 -- 172.21.15.89:0/727148001 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c00bae0 con 0x7faba40b8850 2023-07-06T02:23:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7faba159a700 1 -- 172.21.15.89:0/727148001 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00be50 con 0x7faba40b8850 2023-07-06T02:23:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 >> v1:172.21.15.172:6789/0 conn(0x7fa3b8141870 legacy=0x7fa3b815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a4003af0 0x7fa3a4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a40088e0 0x7fa3a400acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a400b430 0x7fa3a400d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400e140 con 0x7fa3a4003af0 2023-07-06T02:23:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400e2b0 con 0x7fa3a400b430 2023-07-06T02:23:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400e470 con 0x7fa3a40088e0 2023-07-06T02:23:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7f628dffb700 1 -- 172.21.15.89:0/2851805971 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288004440 con 0x7f6298070bc0 2023-07-06T02:23:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7f628dffb700 1 -- 172.21.15.89:0/2851805971 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62880045a0 con 0x7f6298070bc0 2023-07-06T02:23:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.370+0000 7f628dffb700 1 -- 172.21.15.89:0/2851805971 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880038e0 con 0x7f6298070bc0 2023-07-06T02:23:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 -- 172.21.15.89:0/3759547699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc2074d0 msgr2=0x7f20fc16d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc2074d0 0x7f20fc16d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 -- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20fc207100 msgr2=0x7f20fc16d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20fc207100 0x7f20fc16d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 -- 172.21.15.89:0/3759547699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc0c30c0 con 0x7f20fc16e1f0 2023-07-06T02:23:14.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fcea9cad700 1 --2- 172.21.15.89:0/903588143 >> v2:172.21.15.89:3300/0 conn(0x7fcea406bed0 0x7fcea406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3bdc83700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a4003af0 0x7fa3a4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fa3be484700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a400b430 0x7fa3a400d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f2101d47700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc16e1f0 0x7f20fc16e660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20fc070510 tx=0x7f20f400a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c0052f0 con 0x7fd0900ad750 2023-07-06T02:23:14.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fd094cf0700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd07406ad80 0x7fd07406d230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ab90 tx=0x7fd088009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7faba899e700 1 -- 172.21.15.89:0/727148001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40b8850 msgr2=0x7faba406bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7faba899e700 1 --2- 172.21.15.89:0/727148001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40b8850 0x7faba406bed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d90 tx=0x7fab9c00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:14.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.373+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62980c5b90 con 0x7f6298070bc0 2023-07-06T02:23:14.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62981e14a0 con 0x7f6298070bc0 2023-07-06T02:23:14.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fa3be484700 1 -- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a40088e0 msgr2=0x7fa3a400acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fa3be484700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a40088e0 0x7fa3a400acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fa3be484700 1 -- 172.21.15.89:0/2163222150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a4003af0 msgr2=0x7fa3a4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fa3be484700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3a4003af0 0x7fa3a4003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fa3be484700 1 -- 172.21.15.89:0/2163222150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b806af20 con 0x7fa3a400b430 2023-07-06T02:23:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fcea9cad700 1 -- 172.21.15.89:0/903588143 >> v1:172.21.15.172:6789/0 conn(0x7fcea4160d60 legacy=0x7fcea40ce9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fcea9cad700 1 -- 172.21.15.89:0/903588143 >> v1:172.21.15.89:6789/0 conn(0x7fcea406b540 legacy=0x7fcea4164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fcea9cad700 1 -- 172.21.15.89:0/903588143 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea406c7e0 con 0x7fcea406bed0 2023-07-06T02:23:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fcea9cad700 1 --2- 172.21.15.89:0/903588143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406bed0 0x7fcea406c2a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce94009840 tx=0x7fce94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b48d6ab4916a88 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fa3677fe700 1 -- 172.21.15.89:0/971264450 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa378014910 con 0x7fa380070580 2023-07-06T02:23:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7f20f1ffb700 1 -- 172.21.15.89:0/3759547699 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400acf0 con 0x7f20fc16e1f0 2023-07-06T02:23:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7f20f1ffb700 1 -- 172.21.15.89:0/3759547699 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f400ae50 con 0x7f20fc16e1f0 2023-07-06T02:23:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7f20f1ffb700 1 -- 172.21.15.89:0/3759547699 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400e610 con 0x7f20fc16e1f0 2023-07-06T02:23:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7faba899e700 1 -- 172.21.15.89:0/727148001 shutdown_connections 2023-07-06T02:23:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7faba899e700 1 --2- 172.21.15.89:0/727148001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40b8850 0x7faba406bed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7faba899e700 1 --2- 172.21.15.89:0/727148001 >> v2:172.21.15.89:3300/0 conn(0x7faba41610d0 0x7faba40cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7faba899e700 1 -- 172.21.15.89:0/727148001 >> 172.21.15.89:0/727148001 conn(0x7faba40d2e80 msgr2=0x7faba40d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc0c33a0 con 0x7f20fc16e1f0 2023-07-06T02:23:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7faba899e700 1 -- 172.21.15.89:0/727148001 shutdown_connections 2023-07-06T02:23:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7faba899e700 1 -- 172.21.15.89:0/727148001 wait complete. 2023-07-06T02:23:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc167100 con 0x7f20fc16e1f0 2023-07-06T02:23:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7fa3677fe700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06adb0 0x7fa36c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7fa3677fe700 1 -- 172.21.15.89:0/971264450 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa378089e40 con 0x7fa380070580 2023-07-06T02:23:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7fa37e7fc700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06adb0 0x7fa36c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.375+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62981e1300 con 0x7f6298070bc0 2023-07-06T02:23:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.376+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3600052f0 con 0x7fa380070580 2023-07-06T02:23:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.376+0000 7fa37e7fc700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06adb0 0x7fa36c06d260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa38016c230 tx=0x7fa374007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.376+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc0712e0 con 0x7f20fc16e1f0 2023-07-06T02:23:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.374+0000 7fa3be484700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a400b430 0x7fa3a400d890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa3b806e080 tx=0x7fa3a4003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48bedf32b364ad50 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.376+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0007d00 con 0x7fa3a400b430 2023-07-06T02:23:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.376+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b0018070 con 0x7fa3a400b430 2023-07-06T02:23:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.376+0000 7fa3bcc81700 1 -- 172.21.15.89:0/2163222150 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0014950 con 0x7fa3a400b430 2023-07-06T02:23:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7faba899e700 1 Processor -- start 2023-07-06T02:23:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7faba899e700 1 -- start start 2023-07-06T02:23:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7f20f1ffb700 1 -- 172.21.15.89:0/3759547699 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20f400e770 con 0x7f20fc16e1f0 2023-07-06T02:23:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7f20f1ffb700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20e006ad60 0x7f20e006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7fa3bfee7700 1 -- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a400b430 msgr2=0x7fa3a400d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7fa3bfee7700 1 --2- 172.21.15.89:0/2163222150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a400b430 0x7fa3a400d890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa3b806e080 tx=0x7fa3a4003780 comp rx=0 tx=0).stop 2023-07-06T02:23:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7f20f1ffb700 1 -- 172.21.15.89:0/3759547699 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f20f4089710 con 0x7f20fc16e1f0 2023-07-06T02:23:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7fa3be484700 1 -- 172.21.15.89:0/2163222150 reap_dead start 2023-07-06T02:23:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7f2101546700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20e006ad60 0x7f20e006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7fa3bfee7700 1 -- 172.21.15.89:0/2163222150 shutdown_connections 2023-07-06T02:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7fa3bfee7700 1 -- 172.21.15.89:0/2163222150 >> 172.21.15.89:0/2163222150 conn(0x7fa3b80ce870 msgr2=0x7fa3b80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7faba899e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba40b8850 0x7faba40c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.377+0000 7faba899e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40cd970 0x7faba40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba899e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba41610d0 0x7faba40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba899e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40ca530 con 0x7faba41610d0 2023-07-06T02:23:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba899e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40ca810 con 0x7faba40b8850 2023-07-06T02:23:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba899e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba40caaf0 con 0x7faba40cd970 2023-07-06T02:23:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7fa3bfee7700 1 -- 172.21.15.89:0/2163222150 shutdown_connections 2023-07-06T02:23:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7f2101546700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20e006ad60 0x7f20e006d210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000c00 tx=0x7f20ec00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7fa3bfee7700 1 -- 172.21.15.89:0/2163222150 wait complete. 2023-07-06T02:23:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba41610d0 0x7faba40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba41610d0 0x7faba40cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60996/0 (socket says 172.21.15.89:60996) 2023-07-06T02:23:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.379+0000 7fa3bfee7700 1 Processor -- start 2023-07-06T02:23:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 -- 172.21.15.89:0/2148782529 learned_addr learned my addr 172.21.15.89:0/2148782529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba259c700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba40b8850 0x7faba40c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba1d9b700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40cd970 0x7faba40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 -- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40cd970 msgr2=0x7faba40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40cd970 0x7faba40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 -- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba40b8850 msgr2=0x7faba40c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba40b8850 0x7faba40c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 -- 172.21.15.89:0/2148782529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009a70 con 0x7faba41610d0 2023-07-06T02:23:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.378+0000 7faba2d9d700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba41610d0 0x7faba40cb5d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faba415b760 tx=0x7fab9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.379+0000 7fa3bfee7700 1 -- start start 2023-07-06T02:23:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bfee7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a400b430 0x7fa3b81f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bfee7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b81e2220 0x7fa3b81efa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bfee7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b81e2690 0x7fa3b81eff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bfee7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80c1bb0 con 0x7fa3b81e2690 2023-07-06T02:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bfee7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80ca760 con 0x7fa3b81e2220 2023-07-06T02:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bfee7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81f1260 con 0x7fa3a400b430 2023-07-06T02:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3be484700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b81e2690 0x7fa3b81eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bdc83700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a400b430 0x7fa3b81f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3be484700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b81e2690 0x7fa3b81eff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32774/0 (socket says 172.21.15.89:32774) 2023-07-06T02:23:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3be484700 1 -- 172.21.15.89:0/780323695 learned_addr learned my addr 172.21.15.89:0/780323695 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.380+0000 7fa3bd482700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b81e2220 0x7fa3b81efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.381+0000 7fa3be484700 1 -- 172.21.15.89:0/780323695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a400b430 msgr2=0x7fa3b81f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.381+0000 7fa3be484700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a400b430 0x7fa3b81f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.381+0000 7fa3be484700 1 -- 172.21.15.89:0/780323695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b81e2220 msgr2=0x7fa3b81efa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.381+0000 7fa3be484700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b81e2220 0x7fa3b81efa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.381+0000 7fa3be484700 1 -- 172.21.15.89:0/780323695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b81f0650 con 0x7fa3b81e2690 2023-07-06T02:23:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.381+0000 7fa3be484700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b81e2690 0x7fa3b81eff60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3b001b000 tx=0x7fa3b0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.381+0000 7fd08d7fa700 1 -- 172.21.15.89:0/4077302946 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd080055cb0 con 0x7fd0900ad750 2023-07-06T02:23:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.382+0000 7fab9affd700 1 -- 172.21.15.89:0/2148782529 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0045c0 con 0x7faba41610d0 2023-07-06T02:23:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.382+0000 7fcea8cab700 1 -- 172.21.15.89:0/903588143 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94004230 con 0x7fcea406bed0 2023-07-06T02:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.382+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba40cadd0 con 0x7faba41610d0 2023-07-06T02:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.382+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba40ccdd0 con 0x7faba41610d0 2023-07-06T02:23:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.382+0000 7fa3ae7fc700 1 -- 172.21.15.89:0/780323695 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b001f680 con 0x7fa3b81e2690 2023-07-06T02:23:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.383+0000 7f20f1ffb700 1 -- 172.21.15.89:0/3759547699 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f20f4054520 con 0x7f20fc16e1f0 2023-07-06T02:23:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.383+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba40cb2f0 con 0x7faba41610d0 2023-07-06T02:23:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.383+0000 7f628dffb700 1 -- 172.21.15.89:0/2851805971 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6288003a40 con 0x7f6298070bc0 2023-07-06T02:23:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.383+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b81f0930 con 0x7fa3b81e2690 2023-07-06T02:23:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.383+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b81f0ef0 con 0x7fa3b81e2690 2023-07-06T02:23:14.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.383+0000 7f628dffb700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f628406ae90 0x7f628406d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.384+0000 7f629d207700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f628406ae90 0x7f628406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.384+0000 7f629d207700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f628406ae90 0x7f628406d340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f62900113e0 tx=0x7f629000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.384+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b81f1410 con 0x7fa3b81e2690 2023-07-06T02:23:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.388+0000 7f628dffb700 1 -- 172.21.15.89:0/2851805971 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f62880153a0 con 0x7f6298070bc0 2023-07-06T02:23:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.388+0000 7fcea8cab700 1 -- 172.21.15.89:0/903588143 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94004390 con 0x7fcea406bed0 2023-07-06T02:23:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.388+0000 7fcea8cab700 1 -- 172.21.15.89:0/903588143 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940036a0 con 0x7fcea406bed0 2023-07-06T02:23:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.388+0000 7fab9affd700 1 -- 172.21.15.89:0/2148782529 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c004720 con 0x7faba41610d0 2023-07-06T02:23:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.388+0000 7fab9affd700 1 -- 172.21.15.89:0/2148782529 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0093f0 con 0x7faba41610d0 2023-07-06T02:23:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.388+0000 7fa3677fe700 1 -- 172.21.15.89:0/971264450 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa378054be0 con 0x7fa380070580 2023-07-06T02:23:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.389+0000 7fa3ae7fc700 1 -- 172.21.15.89:0/780323695 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b00094d0 con 0x7fa3b81e2690 2023-07-06T02:23:14.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.386+0000 7f8b0e5ec700 1 Processor -- start 2023-07-06T02:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.386+0000 7f8b0e5ec700 1 -- start start 2023-07-06T02:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b0e5ec700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b08166ac0 0x7f8b0816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b0e5ec700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8b080ce2e0 0x7f8b0816d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b0e5ec700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0816a540 con 0x7f8b08070580 2023-07-06T02:23:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b0e5ec700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b080cc5e0 con 0x7f8b08166ac0 2023-07-06T02:23:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b0e5ec700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b080cc8c0 con 0x7f8b080ce2e0 2023-07-06T02:23:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b07fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b08166ac0 0x7f8b0816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b07fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b08166ac0 0x7f8b0816a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32778/0 (socket says 172.21.15.89:32778) 2023-07-06T02:23:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b07fff700 1 -- 172.21.15.89:0/2247008144 learned_addr learned my addr 172.21.15.89:0/2247008144 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.394+0000 7fab9affd700 1 -- 172.21.15.89:0/2148782529 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab9c01c410 con 0x7faba41610d0 2023-07-06T02:23:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.395+0000 7fab9affd700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fab8c06ace0 0x7fab8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.395+0000 7fab9affd700 1 -- 172.21.15.89:0/2148782529 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fab9c08b8c0 con 0x7faba41610d0 2023-07-06T02:23:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.395+0000 7faba259c700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fab8c06ace0 0x7fab8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.396+0000 7faba259c700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fab8c06ace0 0x7fab8c06d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab94007b10 tx=0x7fab94007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.400+0000 7fab9affd700 1 -- 172.21.15.89:0/2148782529 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fab9c05a2c0 con 0x7faba41610d0 2023-07-06T02:23:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.391+0000 7fceabf11700 1 -- 172.21.15.89:0/903588143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406bed0 msgr2=0x7fcea406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.391+0000 7fceabf11700 1 --2- 172.21.15.89:0/903588143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406bed0 0x7fcea406c2a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce94009840 tx=0x7fce94009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.391+0000 7fceabf11700 1 -- 172.21.15.89:0/903588143 shutdown_connections 2023-07-06T02:23:14.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.391+0000 7fceabf11700 1 --2- 172.21.15.89:0/903588143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406bed0 0x7fcea406c2a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.391+0000 7fceabf11700 1 -- 172.21.15.89:0/903588143 >> 172.21.15.89:0/903588143 conn(0x7fcea40d0590 msgr2=0x7fcea40d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.392+0000 7fceabf11700 1 -- 172.21.15.89:0/903588143 shutdown_connections 2023-07-06T02:23:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.392+0000 7fceabf11700 1 -- 172.21.15.89:0/903588143 wait complete. 2023-07-06T02:23:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.389+0000 7fa3ae7fc700 1 -- 172.21.15.89:0/780323695 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b001f9e0 con 0x7fa3b81e2690 2023-07-06T02:23:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.396+0000 7fa3ae7fc700 1 -- 172.21.15.89:0/780323695 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3b001fb40 con 0x7fa3b81e2690 2023-07-06T02:23:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.397+0000 7fa3ae7fc700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa39c06ace0 0x7fa39c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.397+0000 7fa3bdc83700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa39c06ace0 0x7fa39c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.397+0000 7fa3ae7fc700 1 -- 172.21.15.89:0/780323695 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa3b001a070 con 0x7fa3b81e2690 2023-07-06T02:23:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.397+0000 7fa3bdc83700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa39c06ace0 0x7fa39c06d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000c00 tx=0x7fa3b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.405+0000 7fa3ae7fc700 1 -- 172.21.15.89:0/780323695 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3b00560f0 con 0x7fa3b81e2690 2023-07-06T02:23:14.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.393+0000 7f628dffb700 1 -- 172.21.15.89:0/2851805971 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f62880590f0 con 0x7f6298070bc0 2023-07-06T02:23:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.505+0000 7f8b07fff700 1 -- 172.21.15.89:0/2247008144 >> v2:172.21.15.172:3300/0 conn(0x7f8b080ce2e0 msgr2=0x7f8b0816d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:14.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.388+0000 7f8b06ffd700 1 -- 172.21.15.89:0/2247008144 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216962800 0 0) 0x7f8b0816a540 con 0x7f8b08070580 2023-07-06T02:23:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.506+0000 7fceabf11700 1 Processor -- start 2023-07-06T02:23:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.507+0000 7fceabf11700 1 -- start start 2023-07-06T02:23:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fceabf11700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406b540 0x7fcea4158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fceabf11700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcea406bed0 0x7fcea40c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fceabf11700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4160d60 0x7fcea40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fceabf11700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4158a40 con 0x7fcea406b540 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fceabf11700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ca8c0 con 0x7fcea4160d60 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fceabf11700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40caba0 con 0x7fcea406bed0 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406b540 0x7fcea4158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406b540 0x7fcea4158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32788/0 (socket says 172.21.15.89:32788) 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 -- 172.21.15.89:0/2459083846 learned_addr learned my addr 172.21.15.89:0/2459083846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 -- 172.21.15.89:0/2459083846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcea406bed0 msgr2=0x7fcea40c9f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcea406bed0 0x7fcea40c9f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 -- 172.21.15.89:0/2459083846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4160d60 msgr2=0x7fcea40cc5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T02:23:14.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4160d60 0x7fcea40cc5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.508+0000 7fcea9cad700 1 -- 172.21.15.89:0/2459083846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940094a0 con 0x7fcea406b540 2023-07-06T02:23:14.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.387+0000 7f8b077fe700 1 --2- 172.21.15.89:0/2247008144 >> v2:172.21.15.172:3300/0 conn(0x7f8b080ce2e0 0x7f8b0816d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.509+0000 7f8b07fff700 1 --2- 172.21.15.89:0/2247008144 >> v2:172.21.15.172:3300/0 conn(0x7f8b080ce2e0 0x7f8b0816d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.509+0000 7f8b07fff700 1 -- 172.21.15.89:0/2247008144 >> v1:172.21.15.172:6789/0 conn(0x7f8b08070580 legacy=0x7f8b080709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.509+0000 7f8b07fff700 1 -- 172.21.15.89:0/2247008144 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0816ccd0 con 0x7f8b08166ac0 2023-07-06T02:23:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.509+0000 7f8b077fe700 1 --2- 172.21.15.89:0/2247008144 >> v2:172.21.15.172:3300/0 conn(0x7f8b080ce2e0 0x7f8b0816d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:14.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.509+0000 7fcea9cad700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406b540 0x7fcea4158500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fce94009f50 tx=0x7fce94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.509+0000 7f8b07fff700 1 --2- 172.21.15.89:0/2247008144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08166ac0 0x7f8b0816a000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8b0806e5f0 tx=0x7f8af8008c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e41d7bbb0ac627 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.510+0000 7fce9a7fc700 1 -- 172.21.15.89:0/2459083846 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94011830 con 0x7fcea406b540 2023-07-06T02:23:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.510+0000 7fce9a7fc700 1 -- 172.21.15.89:0/2459083846 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94011990 con 0x7fcea406b540 2023-07-06T02:23:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.510+0000 7fce9a7fc700 1 -- 172.21.15.89:0/2459083846 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94011d20 con 0x7fcea406b540 2023-07-06T02:23:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.511+0000 7f8b06ffd700 1 -- 172.21.15.89:0/2247008144 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800dc90 con 0x7f8b08166ac0 2023-07-06T02:23:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.511+0000 7f8b06ffd700 1 -- 172.21.15.89:0/2247008144 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af800ddf0 con 0x7f8b08166ac0 2023-07-06T02:23:14.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.511+0000 7f8b06ffd700 1 -- 172.21.15.89:0/2247008144 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8011650 con 0x7f8b08166ac0 2023-07-06T02:23:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.511+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea40cae80 con 0x7fcea406b540 2023-07-06T02:23:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.511+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea40cb440 con 0x7fcea406b540 2023-07-06T02:23:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.512+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/2247008144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08166ac0 msgr2=0x7f8b0816a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.512+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/2247008144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08166ac0 0x7f8b0816a000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8b0806e5f0 tx=0x7f8af8008c50 comp rx=0 tx=0).stop 2023-07-06T02:23:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.513+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea40cb660 con 0x7fcea406b540 2023-07-06T02:23:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.512+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/2247008144 shutdown_connections 2023-07-06T02:23:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.512+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/2247008144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08166ac0 0x7f8b0816a000 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.512+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/2247008144 >> v2:172.21.15.172:3300/0 conn(0x7f8b080ce2e0 0x7f8b0816d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.512+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/2247008144 >> 172.21.15.89:0/2247008144 conn(0x7f8b080ce870 msgr2=0x7f8b080cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.514+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/2247008144 shutdown_connections 2023-07-06T02:23:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.514+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/2247008144 wait complete. 2023-07-06T02:23:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.516+0000 7f8b0e5ec700 1 Processor -- start 2023-07-06T02:23:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.513+0000 7fce9a7fc700 1 -- 172.21.15.89:0/2459083846 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce94022070 con 0x7fcea406b540 2023-07-06T02:23:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.514+0000 7fce9a7fc700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fce8806ad30 0x7fce8806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.514+0000 7fce9a7fc700 1 -- 172.21.15.89:0/2459083846 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fce9408b460 con 0x7fcea406b540 2023-07-06T02:23:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.517+0000 7f8b0e5ec700 1 -- start start 2023-07-06T02:23:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b0e5ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08070580 0x7f8b080b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b0e5ec700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b080ce2e0 0x7f8b08087f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b0e5ec700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b08166ac0 0x7f8b080c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b0e5ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08088d00 con 0x7f8b08070580 2023-07-06T02:23:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b0e5ec700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080b2890 con 0x7f8b08166ac0 2023-07-06T02:23:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b0e5ec700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080b2b70 con 0x7f8b080ce2e0 2023-07-06T02:23:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b07fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08070580 0x7f8b080b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b07fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08070580 0x7f8b080b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32792/0 (socket says 172.21.15.89:32792) 2023-07-06T02:23:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.518+0000 7f8b07fff700 1 -- 172.21.15.89:0/4281767465 learned_addr learned my addr 172.21.15.89:0/4281767465 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.519+0000 7f8b07fff700 1 -- 172.21.15.89:0/4281767465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b080ce2e0 msgr2=0x7f8b08087f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T02:23:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.519+0000 7f8b07fff700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b080ce2e0 0x7f8b08087f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.519+0000 7f8b07fff700 1 -- 172.21.15.89:0/4281767465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b08166ac0 msgr2=0x7f8b080c5940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T02:23:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.519+0000 7f8b07fff700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b08166ac0 0x7f8b080c5940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.519+0000 7f8b07fff700 1 -- 172.21.15.89:0/4281767465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af80085d0 con 0x7f8b08070580 2023-07-06T02:23:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.519+0000 7f8b07fff700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08070580 0x7f8b080b3930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8af8006f80 tx=0x7f8af8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.520+0000 7fcea94ac700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fce8806ad30 0x7fce8806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.520+0000 7fcea94ac700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fce8806ad30 0x7fce8806d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcea4165690 tx=0x7fcea0006c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.523+0000 7f8b04ff9700 1 -- 172.21.15.89:0/4281767465 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af801f070 con 0x7f8b08070580 2023-07-06T02:23:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.523+0000 7f8b04ff9700 1 -- 172.21.15.89:0/4281767465 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af80038b0 con 0x7f8b08070580 2023-07-06T02:23:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.523+0000 7f8b04ff9700 1 -- 172.21.15.89:0/4281767465 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8003c40 con 0x7f8b08070580 2023-07-06T02:23:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.523+0000 7fce9a7fc700 1 -- 172.21.15.89:0/2459083846 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fce940562d0 con 0x7fcea406b540 2023-07-06T02:23:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.523+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b080b2e50 con 0x7f8b08070580 2023-07-06T02:23:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.523+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b080b3410 con 0x7f8b08070580 2023-07-06T02:23:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.527+0000 7f8b04ff9700 1 -- 172.21.15.89:0/4281767465 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8af801a450 con 0x7f8b08070580 2023-07-06T02:23:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.527+0000 7f8b04ff9700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8af406adb0 0x7f8af406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.527+0000 7f8b04ff9700 1 -- 172.21.15.89:0/4281767465 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8af808ae50 con 0x7f8b08070580 2023-07-06T02:23:14.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.534+0000 7f8b077fe700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8af406adb0 0x7f8af406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:14.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.535+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae80052f0 con 0x7f8b08070580 2023-07-06T02:23:14.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.535+0000 7f8b077fe700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8af406adb0 0x7f8af406d260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006fd0 tx=0x7f8afc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.545+0000 7f8b04ff9700 1 -- 172.21.15.89:0/4281767465 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8af80597c0 con 0x7f8b08070580 2023-07-06T02:23:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.688+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f62981e19c0 con 0x7f6298070bc0 2023-07-06T02:23:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.689+0000 7f628dffb700 1 -- 172.21.15.89:0/2851805971 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f62981e19c0 con 0x7f6298070bc0 2023-07-06T02:23:14.694 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.692+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f628406ae90 msgr2=0x7f628406d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.692+0000 7f629f46b700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f628406ae90 0x7f628406d340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f62900113e0 tx=0x7f629000b040 comp rx=0 tx=0).stop 2023-07-06T02:23:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.692+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298070bc0 msgr2=0x7f6298202ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.692+0000 7f629f46b700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298070bc0 0x7f6298202ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6288000c00 tx=0x7f6288004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 shutdown_connections 2023-07-06T02:23:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62980c0f10 0x7f62980c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62980707f0 0x7f62981e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f628406ae90 0x7f628406d340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 --2- 172.21.15.89:0/2851805971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6298070bc0 0x7f6298202ee0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 >> 172.21.15.89:0/2851805971 conn(0x7f62980ce870 msgr2=0x7f62980c4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 shutdown_connections 2023-07-06T02:23:14.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.693+0000 7f629f46b700 1 -- 172.21.15.89:0/2851805971 wait complete. 2023-07-06T02:23:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.699+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa3b81f1410 con 0x7fa3b81e2690 2023-07-06T02:23:14.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.699+0000 7fa3ae7fc700 1 -- 172.21.15.89:0/780323695 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3b81f1410 con 0x7fa3b81e2690 2023-07-06T02:23:14.705 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-07-06T02:23:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.703+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa39c06ace0 msgr2=0x7fa39c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.703+0000 7fa3bfee7700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa39c06ace0 0x7fa39c06d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000c00 tx=0x7fa3b4009040 comp rx=0 tx=0).stop 2023-07-06T02:23:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.703+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b81e2690 msgr2=0x7fa3b81eff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.703+0000 7fa3bfee7700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b81e2690 0x7fa3b81eff60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3b001b000 tx=0x7fa3b0004b40 comp rx=0 tx=0).stop 2023-07-06T02:23:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 shutdown_connections 2023-07-06T02:23:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa39c06ace0 0x7fa39c06d190 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000c00 tx=0x7fa3b4009040 comp rx=0 tx=0).stop 2023-07-06T02:23:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3b81e2690 0x7fa3b81eff60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3a400b430 0x7fa3b81f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 --2- 172.21.15.89:0/780323695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b81e2220 0x7fa3b81efa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 >> 172.21.15.89:0/780323695 conn(0x7fa3b80ce870 msgr2=0x7fa3b80b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 shutdown_connections 2023-07-06T02:23:14.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.704+0000 7fa3bfee7700 1 -- 172.21.15.89:0/780323695 wait complete. 2023-07-06T02:23:14.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-07-06T02:23:14.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2023-07-06T02:23:14.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.772+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa3600055a0 con 0x7fa380070580 2023-07-06T02:23:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.773+0000 7fa3677fe700 1 -- 172.21.15.89:0/971264450 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa378058200 con 0x7fa380070580 2023-07-06T02:23:14.776 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-07-06T02:23:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.776+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06adb0 msgr2=0x7fa36c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.776+0000 7fa3857db700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06adb0 0x7fa36c06d260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa38016c230 tx=0x7fa374007dc0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.776+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa380070580 msgr2=0x7fa38007f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.776+0000 7fa3857db700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa380070580 0x7fa38007f370 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa378007e60 tx=0x7fa37800e3c0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 shutdown_connections 2023-07-06T02:23:14.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38016d4d0 0x7fa3800c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06adb0 0x7fa36c06d260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa380070580 0x7fa38007f370 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 --2- 172.21.15.89:0/971264450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3800ce2e0 0x7fa38007f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 >> 172.21.15.89:0/971264450 conn(0x7fa3800ce870 msgr2=0x7fa38006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 shutdown_connections 2023-07-06T02:23:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.777+0000 7fa3857db700 1 -- 172.21.15.89:0/971264450 wait complete. 2023-07-06T02:23:14.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.785+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd06c0055a0 con 0x7fd0900ad750 2023-07-06T02:23:14.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.785+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f20fc074270 con 0x7f20fc16e1f0 2023-07-06T02:23:14.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.786+0000 7fd08d7fa700 1 -- 172.21.15.89:0/4077302946 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd0800592d0 con 0x7fd0900ad750 2023-07-06T02:23:14.789 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-07-06T02:23:14.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.786+0000 7f20f1ffb700 1 -- 172.21.15.89:0/3759547699 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20f4057b40 con 0x7f20fc16e1f0 2023-07-06T02:23:14.790 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-07-06T02:23:14.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.788+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faba40cb2f0 con 0x7faba41610d0 2023-07-06T02:23:14.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.788+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd07406ad80 msgr2=0x7fd07406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.788+0000 7fd096f54700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd07406ad80 0x7fd07406d230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ab90 tx=0x7fd088009250 comp rx=0 tx=0).stop 2023-07-06T02:23:14.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.788+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900ad750 msgr2=0x7fd0900b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.788+0000 7fd096f54700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900ad750 0x7fd0900b3e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd09006d7a0 tx=0x7fd080004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 shutdown_connections 2023-07-06T02:23:14.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd090163e60 0x7fd0901493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd09006a060 0x7fd09015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd07406ad80 0x7fd07406d230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 --2- 172.21.15.89:0/4077302946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900ad750 0x7fd0900b3e70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 >> 172.21.15.89:0/4077302946 conn(0x7fd0900ce870 msgr2=0x7fd090151ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 shutdown_connections 2023-07-06T02:23:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20e006ad60 msgr2=0x7f20e006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7f21037aa700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20e006ad60 0x7f20e006d210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000c00 tx=0x7f20ec00a000 comp rx=0 tx=0).stop 2023-07-06T02:23:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc16e1f0 msgr2=0x7f20fc16e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7f21037aa700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc16e1f0 0x7f20fc16e660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20fc070510 tx=0x7f20f400a570 comp rx=0 tx=0).stop 2023-07-06T02:23:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.789+0000 7fd096f54700 1 -- 172.21.15.89:0/4077302946 wait complete. 2023-07-06T02:23:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 shutdown_connections 2023-07-06T02:23:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc16e1f0 0x7f20fc16e660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20e006ad60 0x7f20e006d210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20fc207100 0x7f20fc16d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 --2- 172.21.15.89:0/3759547699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc2074d0 0x7f20fc16d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 >> 172.21.15.89:0/3759547699 conn(0x7f20fc0ce870 msgr2=0x7f20fc16bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 shutdown_connections 2023-07-06T02:23:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7f21037aa700 1 -- 172.21.15.89:0/3759547699 wait complete. 2023-07-06T02:23:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.790+0000 7fab9affd700 1 -- 172.21.15.89:0/2148782529 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faba40cb2f0 con 0x7faba41610d0 2023-07-06T02:23:14.796 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.792+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fab8c06ace0 msgr2=0x7fab8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.792+0000 7faba899e700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fab8c06ace0 0x7fab8c06d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab94007b10 tx=0x7fab94007a60 comp rx=0 tx=0).stop 2023-07-06T02:23:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.792+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba41610d0 msgr2=0x7faba40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.792+0000 7faba899e700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba41610d0 0x7faba40cb5d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faba415b760 tx=0x7fab9c004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.793+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 shutdown_connections 2023-07-06T02:23:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.793+0000 7faba899e700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fab8c06ace0 0x7fab8c06d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.793+0000 7faba899e700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba41610d0 0x7faba40cb5d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.793+0000 7faba899e700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba40b8850 0x7faba40c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.793+0000 7faba899e700 1 --2- 172.21.15.89:0/2148782529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba40cd970 0x7faba40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.793+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 >> 172.21.15.89:0/2148782529 conn(0x7faba40d2e80 msgr2=0x7faba4163c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-07-06T02:23:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.795+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 shutdown_connections 2023-07-06T02:23:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.795+0000 7faba899e700 1 -- 172.21.15.89:0/2148782529 wait complete. 2023-07-06T02:23:14.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-07-06T02:23:14.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-07-06T02:23:14.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-07-06T02:23:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.831+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcea40cb660 con 0x7fcea406b540 2023-07-06T02:23:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.831+0000 7fce9a7fc700 1 -- 172.21.15.89:0/2459083846 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce940598f0 con 0x7fcea406b540 2023-07-06T02:23:14.834 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:14.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fce8806ad30 msgr2=0x7fce8806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fce8806ad30 0x7fce8806d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcea4165690 tx=0x7fcea0006c60 comp rx=0 tx=0).stop 2023-07-06T02:23:14.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406b540 msgr2=0x7fcea4158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406b540 0x7fcea4158500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fce94009f50 tx=0x7fce94004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 shutdown_connections 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4160d60 0x7fcea40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fce8806ad30 0x7fce8806d1e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcea406b540 0x7fcea4158500 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 --2- 172.21.15.89:0/2459083846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcea406bed0 0x7fcea40c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 >> 172.21.15.89:0/2459083846 conn(0x7fcea40d0590 msgr2=0x7fcea40d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 shutdown_connections 2023-07-06T02:23:14.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.833+0000 7fceabf11700 1 -- 172.21.15.89:0/2459083846 wait complete. 2023-07-06T02:23:14.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2023-07-06T02:23:14.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.848+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8ae80055a0 con 0x7f8b08070580 2023-07-06T02:23:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.848+0000 7f8b04ff9700 1 -- 172.21.15.89:0/4281767465 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8af8028070 con 0x7f8b08070580 2023-07-06T02:23:14.854 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-07-06T02:23:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8af406adb0 msgr2=0x7f8af406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8af406adb0 0x7f8af406d260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006fd0 tx=0x7f8afc008040 comp rx=0 tx=0).stop 2023-07-06T02:23:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08070580 msgr2=0x7f8b080b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08070580 0x7f8b080b3930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8af8006f80 tx=0x7f8af8004750 comp rx=0 tx=0).stop 2023-07-06T02:23:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 shutdown_connections 2023-07-06T02:23:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b08166ac0 0x7f8b080c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8af406adb0 0x7f8af406d260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b08070580 0x7f8b080b3930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 --2- 172.21.15.89:0/4281767465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b080ce2e0 0x7f8b08087f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 >> 172.21.15.89:0/4281767465 conn(0x7f8b080ce870 msgr2=0x7f8b08168710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 shutdown_connections 2023-07-06T02:23:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:14.850+0000 7f8b0e5ec700 1 -- 172.21.15.89:0/4281767465 wait complete. 2023-07-06T02:23:14.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-07-06T02:23:15.721 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T02:23:15.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T02:23:15.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T02:23:15.809 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T02:23:15.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T02:23:15.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T02:23:15.849 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T02:23:15.866 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T02:23:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.869+0000 7f349f8c6700 1 Processor -- start 2023-07-06T02:23:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.869+0000 7f349f8c6700 1 -- start start 2023-07-06T02:23:15.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.870+0000 7f349f8c6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34980707f0 0x7f349806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.870+0000 7f349f8c6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349806a960 con 0x7f3498141870 2023-07-06T02:23:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.870+0000 7f349f8c6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349806ac40 con 0x7f3498070bc0 2023-07-06T02:23:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.870+0000 7f349f8c6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34980cd100 con 0x7f34980707f0 2023-07-06T02:23:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.871+0000 7f349de63700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3498070bc0 0x7f349806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33460/0 (socket says 172.21.15.89:33460) 2023-07-06T02:23:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.871+0000 7f349de63700 1 -- 172.21.15.89:0/4282889360 learned_addr learned my addr 172.21.15.89:0/4282889360 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:15.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.871+0000 7f349d662700 1 --2- 172.21.15.89:0/4282889360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34980707f0 0x7f349806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.871+0000 7f348ffff700 1 -- 172.21.15.89:0/4282889360 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474040199 0 0) 0x7f349806ac40 con 0x7f3498070bc0 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.871+0000 7f348ffff700 1 -- 172.21.15.89:0/4282889360 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34840035b0 con 0x7f3498070bc0 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.872+0000 7f349d662700 1 -- 172.21.15.89:0/4282889360 >> v1:172.21.15.172:6789/0 conn(0x7f3498070bc0 legacy=0x7f349806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.872+0000 7f349d662700 1 -- 172.21.15.89:0/4282889360 >> v1:172.21.15.89:6789/0 conn(0x7f3498141870 legacy=0x7f349806cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.872+0000 7f349d662700 1 -- 172.21.15.89:0/4282889360 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349806af20 con 0x7f34980707f0 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.872+0000 7f349d662700 1 --2- 172.21.15.89:0/4282889360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34980707f0 0x7f349806a060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3488009870 tx=0x7f3488009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1bd65932bbc4896 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.872+0000 7f348ffff700 1 -- 172.21.15.89:0/4282889360 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488004230 con 0x7f34980707f0 2023-07-06T02:23:15.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.872+0000 7f348ffff700 1 -- 172.21.15.89:0/4282889360 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3488004390 con 0x7f34980707f0 2023-07-06T02:23:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.872+0000 7f348ffff700 1 -- 172.21.15.89:0/4282889360 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34880036a0 con 0x7f34980707f0 2023-07-06T02:23:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.873+0000 7f349f8c6700 1 -- 172.21.15.89:0/4282889360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34980707f0 msgr2=0x7f349806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.873+0000 7f349f8c6700 1 --2- 172.21.15.89:0/4282889360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34980707f0 0x7f349806a060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3488009870 tx=0x7f3488009b80 comp rx=0 tx=0).stop 2023-07-06T02:23:15.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.876+0000 7f349f8c6700 1 -- 172.21.15.89:0/4282889360 shutdown_connections 2023-07-06T02:23:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.876+0000 7f349f8c6700 1 --2- 172.21.15.89:0/4282889360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34980707f0 0x7f349806a060 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.876+0000 7f349f8c6700 1 -- 172.21.15.89:0/4282889360 >> 172.21.15.89:0/4282889360 conn(0x7f34980ce870 msgr2=0x7f34980cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.876+0000 7f349f8c6700 1 -- 172.21.15.89:0/4282889360 shutdown_connections 2023-07-06T02:23:15.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.876+0000 7f349f8c6700 1 -- 172.21.15.89:0/4282889360 wait complete. 2023-07-06T02:23:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.878+0000 7f349f8c6700 1 Processor -- start 2023-07-06T02:23:15.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.878+0000 7f349f8c6700 1 -- start start 2023-07-06T02:23:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.879+0000 7f349f8c6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34980707f0 0x7f34981f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.879+0000 7f349f8c6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3498070bc0 0x7f34980c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.879+0000 7f349f8c6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3498141870 0x7f34980c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.879+0000 7f349f8c6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980c3260 con 0x7f3498141870 2023-07-06T02:23:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.879+0000 7f349f8c6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980c3540 con 0x7f34980707f0 2023-07-06T02:23:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.879+0000 7f349f8c6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34980c3820 con 0x7f3498070bc0 2023-07-06T02:23:15.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3498141870 0x7f34980c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3498141870 0x7f34980c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32810/0 (socket says 172.21.15.89:32810) 2023-07-06T02:23:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 -- 172.21.15.89:0/3173105200 learned_addr learned my addr 172.21.15.89:0/3173105200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349d662700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34980707f0 0x7f34981f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349ce61700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3498070bc0 0x7f34980c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 -- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3498070bc0 msgr2=0x7f34980c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:15.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3498070bc0 0x7f34980c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 -- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34980707f0 msgr2=0x7f34981f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:15.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34980707f0 0x7f34981f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.880+0000 7f349de63700 1 -- 172.21.15.89:0/3173105200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34880094a0 con 0x7f3498141870 2023-07-06T02:23:15.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.881+0000 7f349de63700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3498141870 0x7f34980c4880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f349000cb10 tx=0x7f349000ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.882+0000 7f348dffb700 1 -- 172.21.15.89:0/3173105200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000a9e0 con 0x7f3498141870 2023-07-06T02:23:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.882+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34980c3b00 con 0x7f3498141870 2023-07-06T02:23:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.882+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34980c24f0 con 0x7f3498141870 2023-07-06T02:23:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.883+0000 7f348dffb700 1 -- 172.21.15.89:0/3173105200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000ab40 con 0x7f3498141870 2023-07-06T02:23:15.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.883+0000 7f348dffb700 1 -- 172.21.15.89:0/3173105200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490012400 con 0x7f3498141870 2023-07-06T02:23:15.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.884+0000 7f348dffb700 1 -- 172.21.15.89:0/3173105200 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34900125e0 con 0x7f3498141870 2023-07-06T02:23:15.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.885+0000 7f348dffb700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f347c06ace0 0x7f347c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.885+0000 7f348dffb700 1 -- 172.21.15.89:0/3173105200 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f349005bb50 con 0x7f3498141870 2023-07-06T02:23:15.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.885+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34780052f0 con 0x7f3498141870 2023-07-06T02:23:15.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.885+0000 7f349d662700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f347c06ace0 0x7f347c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.887+0000 7f349d662700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f347c06ace0 0x7f347c06d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3488000c00 tx=0x7f348801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:15.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.894+0000 7f348dffb700 1 -- 172.21.15.89:0/3173105200 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f349008e060 con 0x7f3498141870 2023-07-06T02:23:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.952+0000 7f05ad0dc700 1 Processor -- start 2023-07-06T02:23:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.952+0000 7f05ad0dc700 1 -- start start 2023-07-06T02:23:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.953+0000 7f05ad0dc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05a8070830 0x7f05a80ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.953+0000 7f05ad0dc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a8164e20 0x7f05a8070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.953+0000 7f05ad0dc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a806b7e0 con 0x7f05a806c710 2023-07-06T02:23:15.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.953+0000 7f05ad0dc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80ce8e0 con 0x7f05a8070830 2023-07-06T02:23:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.953+0000 7f05ad0dc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8160d90 con 0x7f05a8164e20 2023-07-06T02:23:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a7fff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05a8070830 0x7f05a80ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a7fff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05a8070830 0x7f05a80ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59806/0 (socket says 172.21.15.89:59806) 2023-07-06T02:23:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a7fff700 1 -- 172.21.15.89:0/3603256500 learned_addr learned my addr 172.21.15.89:0/3603256500 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:15.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a77fe700 1 --2- 172.21.15.89:0/3603256500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a8164e20 0x7f05a8070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a77fe700 1 -- 172.21.15.89:0/3603256500 >> v2:172.21.15.172:3300/0 conn(0x7f05a8070830 msgr2=0x7f05a80ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a77fe700 1 --2- 172.21.15.89:0/3603256500 >> v2:172.21.15.172:3300/0 conn(0x7f05a8070830 0x7f05a80ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a77fe700 1 -- 172.21.15.89:0/3603256500 >> v1:172.21.15.172:6789/0 conn(0x7f05a806c710 legacy=0x7f05a806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:15.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.954+0000 7f05a77fe700 1 -- 172.21.15.89:0/3603256500 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a806a060 con 0x7f05a8164e20 2023-07-06T02:23:15.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.963+0000 7f05a77fe700 1 --2- 172.21.15.89:0/3603256500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a8164e20 0x7f05a8070c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0598009840 tx=0x7f0598009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476c1c4ff7be6efc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:15.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.964+0000 7f05a6ffd700 1 -- 172.21.15.89:0/3603256500 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598004230 con 0x7f05a8164e20 2023-07-06T02:23:15.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.964+0000 7f05a6ffd700 1 -- 172.21.15.89:0/3603256500 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0598004390 con 0x7f05a8164e20 2023-07-06T02:23:15.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.964+0000 7f05a6ffd700 1 -- 172.21.15.89:0/3603256500 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980036a0 con 0x7f05a8164e20 2023-07-06T02:23:15.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3603256500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a8164e20 msgr2=0x7f05a8070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:15.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3603256500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a8164e20 0x7f05a8070c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0598009840 tx=0x7f0598009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:15.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3603256500 shutdown_connections 2023-07-06T02:23:15.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3603256500 >> v2:172.21.15.172:3300/0 conn(0x7f05a8070830 0x7f05a80ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3603256500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a8164e20 0x7f05a8070c00 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0598009840 tx=0x7f0598009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:15.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3603256500 >> 172.21.15.89:0/3603256500 conn(0x7f05a80ceaf0 msgr2=0x7f05a80ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:15.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3603256500 shutdown_connections 2023-07-06T02:23:15.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.965+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3603256500 wait complete. 2023-07-06T02:23:15.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.967+0000 7f05ad0dc700 1 Processor -- start 2023-07-06T02:23:15.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.968+0000 7f05ad0dc700 1 -- start start 2023-07-06T02:23:15.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05ad0dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05a806c710 0x7f05a814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05ad0dc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a8070830 0x7f05a814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05ad0dc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a80c25b0 0x7f05a8148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05ad0dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c7980 con 0x7f05a806c710 2023-07-06T02:23:15.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05ad0dc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c5450 con 0x7f05a8070830 2023-07-06T02:23:15.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05ad0dc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c4e90 con 0x7f05a80c25b0 2023-07-06T02:23:15.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05a806c710 0x7f05a814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05a7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05a806c710 0x7f05a814e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32824/0 (socket says 172.21.15.89:32824) 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.969+0000 7f05a7fff700 1 -- 172.21.15.89:0/3423390050 learned_addr learned my addr 172.21.15.89:0/3423390050 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.970+0000 7f05a7fff700 1 -- 172.21.15.89:0/3423390050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a80c25b0 msgr2=0x7f05a8148090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.970+0000 7f05a7fff700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a80c25b0 0x7f05a8148090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.970+0000 7f05a7fff700 1 -- 172.21.15.89:0/3423390050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a8070830 msgr2=0x7f05a814e550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.970+0000 7f05a7fff700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a8070830 0x7f05a814e550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.970+0000 7f05a7fff700 1 -- 172.21.15.89:0/3423390050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980094a0 con 0x7f05a806c710 2023-07-06T02:23:15.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.970+0000 7f05a7fff700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05a806c710 0x7f05a814e010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05a00070b0 tx=0x7f05a0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:15.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.971+0000 7f05a4ff9700 1 -- 172.21.15.89:0/3423390050 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000db10 con 0x7f05a806c710 2023-07-06T02:23:15.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.971+0000 7f05a4ff9700 1 -- 172.21.15.89:0/3423390050 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a000f040 con 0x7f05a806c710 2023-07-06T02:23:15.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.971+0000 7f05a4ff9700 1 -- 172.21.15.89:0/3423390050 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0015400 con 0x7f05a806c710 2023-07-06T02:23:15.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.971+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a80c5170 con 0x7f05a806c710 2023-07-06T02:23:15.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.971+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a80c5bc0 con 0x7f05a806c710 2023-07-06T02:23:15.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.974+0000 7f05a4ff9700 1 -- 172.21.15.89:0/3423390050 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f05a0015560 con 0x7f05a806c710 2023-07-06T02:23:15.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.975+0000 7f05a4ff9700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f059406adb0 0x7f059406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.975+0000 7f05a4ff9700 1 -- 172.21.15.89:0/3423390050 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f05a00894e0 con 0x7f05a806c710 2023-07-06T02:23:15.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.978+0000 7f05a77fe700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f059406adb0 0x7f059406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.979+0000 7f05a77fe700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f059406adb0 0x7f059406d260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cc580 tx=0x7f0598008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.985+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05880052f0 con 0x7f05a806c710 2023-07-06T02:23:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.989+0000 7f19bfdd5700 1 Processor -- start 2023-07-06T02:23:15.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.990+0000 7f19bfdd5700 1 -- start start 2023-07-06T02:23:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19bfdd5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b8141870 0x7f19b815b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19bfdd5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19b806dcb0 0x7f19b815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19bfdd5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b806a960 con 0x7f19b80707f0 2023-07-06T02:23:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19bfdd5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b806a770 con 0x7f19b806dcb0 2023-07-06T02:23:15.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19bfdd5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80cd100 con 0x7f19b8141870 2023-07-06T02:23:15.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19be372700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19b806dcb0 0x7f19b815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:15.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19be372700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f19b806dcb0 0x7f19b815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32828/0 (socket says 172.21.15.89:32828) 2023-07-06T02:23:15.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.991+0000 7f19be372700 1 -- 172.21.15.89:0/743399175 learned_addr learned my addr 172.21.15.89:0/743399175 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:15.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.992+0000 7f19be372700 1 -- 172.21.15.89:0/743399175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b8141870 msgr2=0x7f19b815b930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:23:15.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.992+0000 7f19be372700 1 --2- 172.21.15.89:0/743399175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b8141870 0x7f19b815b930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:15.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.992+0000 7f19be372700 1 -- 172.21.15.89:0/743399175 >> v1:172.21.15.89:6789/0 conn(0x7f19b80707f0 legacy=0x7f19b806a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:23:15.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.992+0000 7f19be372700 1 -- 172.21.15.89:0/743399175 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b806ac40 con 0x7f19b806dcb0 2023-07-06T02:23:15.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.992+0000 7f19be372700 1 --2- 172.21.15.89:0/743399175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b806dcb0 0x7f19b815be70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d370 tx=0x7f19b000d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8081af158252edd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:15.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.995+0000 7f19bcb6f700 1 -- 172.21.15.89:0/743399175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0010070 con 0x7f19b806dcb0 2023-07-06T02:23:15.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.996+0000 7f19bcb6f700 1 -- 172.21.15.89:0/743399175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b000dcd0 con 0x7f19b806dcb0 2023-07-06T02:23:15.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.996+0000 7f19bcb6f700 1 -- 172.21.15.89:0/743399175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0007d70 con 0x7f19b806dcb0 2023-07-06T02:23:15.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.996+0000 7f05a4ff9700 1 -- 172.21.15.89:0/3423390050 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f05a0054330 con 0x7f05a806c710 2023-07-06T02:23:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.997+0000 7f19bfdd5700 1 -- 172.21.15.89:0/743399175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b806dcb0 msgr2=0x7f19b815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.997+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/743399175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b806dcb0 0x7f19b815be70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d370 tx=0x7f19b000d680 comp rx=0 tx=0).stop 2023-07-06T02:23:16.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.997+0000 7f19bfdd5700 1 -- 172.21.15.89:0/743399175 shutdown_connections 2023-07-06T02:23:16.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.997+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/743399175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b806dcb0 0x7f19b815be70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.997+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/743399175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b8141870 0x7f19b815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.997+0000 7f19bfdd5700 1 -- 172.21.15.89:0/743399175 >> 172.21.15.89:0/743399175 conn(0x7f19b80ce870 msgr2=0x7f19b80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.999+0000 7f19bfdd5700 1 -- 172.21.15.89:0/743399175 shutdown_connections 2023-07-06T02:23:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:15.999+0000 7f19bfdd5700 1 -- 172.21.15.89:0/743399175 wait complete. 2023-07-06T02:23:16.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.001+0000 7f19bfdd5700 1 Processor -- start 2023-07-06T02:23:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.001+0000 7f19bfdd5700 1 -- start start 2023-07-06T02:23:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19bfdd5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19b806dcb0 0x7f19b80ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19bfdd5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b80707f0 0x7f19b80c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19bfdd5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b8141870 0x7f19b80c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19bfdd5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c7800 con 0x7f19b8141870 2023-07-06T02:23:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19bfdd5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c3620 con 0x7f19b806dcb0 2023-07-06T02:23:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19bfdd5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c6f60 con 0x7f19b80707f0 2023-07-06T02:23:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19be372700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b8141870 0x7f19b80c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19be372700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b8141870 0x7f19b80c1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32832/0 (socket says 172.21.15.89:32832) 2023-07-06T02:23:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19be372700 1 -- 172.21.15.89:0/3461238891 learned_addr learned my addr 172.21.15.89:0/3461238891 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.003+0000 7f19bd370700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b80707f0 0x7f19b80c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19be372700 1 -- 172.21.15.89:0/3461238891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b80707f0 msgr2=0x7f19b80c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19be372700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b80707f0 0x7f19b80c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19be372700 1 -- 172.21.15.89:0/3461238891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19b806dcb0 msgr2=0x7f19b80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19be372700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19b806dcb0 0x7f19b80ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19be372700 1 -- 172.21.15.89:0/3461238891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b000d050 con 0x7f19b8141870 2023-07-06T02:23:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19be372700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b8141870 0x7f19b80c1360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d6c0 tx=0x7f19b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19ae7fc700 1 -- 172.21.15.89:0/3461238891 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b00045a0 con 0x7f19b8141870 2023-07-06T02:23:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19ae7fc700 1 -- 172.21.15.89:0/3461238891 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b0004700 con 0x7f19b8141870 2023-07-06T02:23:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.004+0000 7f19ae7fc700 1 -- 172.21.15.89:0/3461238891 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0008030 con 0x7f19b8141870 2023-07-06T02:23:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.005+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b80c7240 con 0x7f19b8141870 2023-07-06T02:23:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.005+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b80ca840 con 0x7f19b8141870 2023-07-06T02:23:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.009+0000 7f19ae7fc700 1 -- 172.21.15.89:0/3461238891 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f19b0008190 con 0x7f19b8141870 2023-07-06T02:23:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.009+0000 7f19ae7fc700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f19a406ae40 0x7f19a406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.010+0000 7f19ae7fc700 1 -- 172.21.15.89:0/3461238891 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f19b000de40 con 0x7f19b8141870 2023-07-06T02:23:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.010+0000 7f19bdb71700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f19a406ae40 0x7f19a406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.011+0000 7f19bdb71700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f19a406ae40 0x7f19a406d2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009770 tx=0x7f19a8006cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.011+0000 7fb7ca4c8700 1 Processor -- start 2023-07-06T02:23:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.011+0000 7fb7ca4c8700 1 -- start start 2023-07-06T02:23:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.012+0000 7fb7ca4c8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7c406dcb0 0x7fb7c415c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.012+0000 7fb7ca4c8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c406a960 con 0x7fb7c40707f0 2023-07-06T02:23:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.012+0000 7fb7ca4c8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c406ac40 con 0x7fb7c4141870 2023-07-06T02:23:16.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.012+0000 7fb7ca4c8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c406a770 con 0x7fb7c406dcb0 2023-07-06T02:23:16.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c8a65700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7c406dcb0 0x7fb7c415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c8a65700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7c406dcb0 0x7fb7c415c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59830/0 (socket says 172.21.15.89:59830) 2023-07-06T02:23:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c8a65700 1 -- 172.21.15.89:0/824165410 learned_addr learned my addr 172.21.15.89:0/824165410 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c37fe700 1 --1- 172.21.15.89:0/824165410 >> v1:172.21.15.172:6789/0 conn(0x7fb7c4141870 0x7fb7c415b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33496/0 (socket says 172.21.15.89:33496) 2023-07-06T02:23:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741771884 0 0) 0x7fb7c406a960 con 0x7fb7c40707f0 2023-07-06T02:23:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7ac0035b0 con 0x7fb7c40707f0 2023-07-06T02:23:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2059887671 0 0) 0x7fb7ac0035b0 con 0x7fb7c40707f0 2023-07-06T02:23:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 >> v2:172.21.15.172:3300/0 conn(0x7fb7c406dcb0 msgr2=0x7fb7c415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 --2- 172.21.15.89:0/824165410 >> v2:172.21.15.172:3300/0 conn(0x7fb7c406dcb0 0x7fb7c415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 >> v1:172.21.15.172:6789/0 conn(0x7fb7c4141870 legacy=0x7fb7c415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c406af20 con 0x7fb7c40707f0 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.013+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fb7b8001670 con 0x7fb7c40707f0 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 >> v1:172.21.15.89:6789/0 conn(0x7fb7c40707f0 legacy=0x7fb7c406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c2ffd700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7ac003a00 0x7fb7ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c2ffd700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7ac004850 0x7fb7ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c2ffd700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7ac005200 0x7fb7ac00fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0059e0 con 0x7fb7ac004850 2023-07-06T02:23:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac005b20 con 0x7fb7ac005200 2023-07-06T02:23:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac005ce0 con 0x7fb7ac003a00 2023-07-06T02:23:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c3fff700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7ac003a00 0x7fb7ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c3fff700 1 -- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7ac005200 msgr2=0x7fb7ac00fe20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:23:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c3fff700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7ac005200 0x7fb7ac00fe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c3fff700 1 -- 172.21.15.89:0/824165410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7ac004850 msgr2=0x7fb7ac004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:23:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c3fff700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7ac004850 0x7fb7ac004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.014+0000 7fb7c3fff700 1 -- 172.21.15.89:0/824165410 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c406af20 con 0x7fb7ac003a00 2023-07-06T02:23:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.015+0000 7fb7c3fff700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7ac003a00 0x7fb7ac003e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800c260 tx=0x7fb7ac005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6396cf7c75c65de3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.015+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800ce00 con 0x7fb7ac003a00 2023-07-06T02:23:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.015+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8012420 con 0x7fb7ac003a00 2023-07-06T02:23:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.015+0000 7fb7c2ffd700 1 -- 172.21.15.89:0/824165410 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80127d0 con 0x7fb7ac003a00 2023-07-06T02:23:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.016+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7ac003a00 msgr2=0x7fb7ac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.016+0000 7fb7ca4c8700 1 --2- 172.21.15.89:0/824165410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7ac003a00 0x7fb7ac003e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800c260 tx=0x7fb7ac005880 comp rx=0 tx=0).stop 2023-07-06T02:23:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.020+0000 7fb7c8a65700 1 -- 172.21.15.89:0/824165410 reap_dead start 2023-07-06T02:23:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.020+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a00052f0 con 0x7f19b8141870 2023-07-06T02:23:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.024+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/824165410 shutdown_connections 2023-07-06T02:23:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.024+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/824165410 >> 172.21.15.89:0/824165410 conn(0x7fb7c40ce870 msgr2=0x7fb7c40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.024+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/824165410 shutdown_connections 2023-07-06T02:23:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.025+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/824165410 wait complete. 2023-07-06T02:23:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.026+0000 7fb7ca4c8700 1 Processor -- start 2023-07-06T02:23:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.027+0000 7fb7ca4c8700 1 -- start start 2023-07-06T02:23:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.028+0000 7fb7ca4c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40c3060 0x7fb7c40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.028+0000 7fb7ca4c8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c40ca660 0x7fb7c40c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.028+0000 7fb7ca4c8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c40c3970 0x7fb7c41e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.028+0000 7fb7ca4c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c41e2220 con 0x7fb7c40c3060 2023-07-06T02:23:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.028+0000 7fb7ca4c8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c41e3040 con 0x7fb7c40c3970 2023-07-06T02:23:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.028+0000 7fb7ca4c8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40c15b0 con 0x7fb7c40ca660 2023-07-06T02:23:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40c3060 0x7fb7c40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40c3060 0x7fb7c40ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32846/0 (socket says 172.21.15.89:32846) 2023-07-06T02:23:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 -- 172.21.15.89:0/1832647775 learned_addr learned my addr 172.21.15.89:0/1832647775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 -- 172.21.15.89:0/1832647775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c40ca660 msgr2=0x7fb7c40c3430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T02:23:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c40ca660 0x7fb7c40c3430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 -- 172.21.15.89:0/1832647775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c40c3970 msgr2=0x7fb7c41e34d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T02:23:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c40c3970 0x7fb7c41e34d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 -- 172.21.15.89:0/1832647775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c41e2430 con 0x7fb7c40c3060 2023-07-06T02:23:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7fb7c3fff700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40c3060 0x7fb7c40ca120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8014040 tx=0x7fb7b8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.029+0000 7f19ae7fc700 1 -- 172.21.15.89:0/3461238891 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f19b0056350 con 0x7f19b8141870 2023-07-06T02:23:16.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.030+0000 7fb7c0ff9700 1 -- 172.21.15.89:0/1832647775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80052c0 con 0x7fb7c40c3060 2023-07-06T02:23:16.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.030+0000 7fb7c0ff9700 1 -- 172.21.15.89:0/1832647775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8020070 con 0x7fb7c40c3060 2023-07-06T02:23:16.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.030+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c41e2710 con 0x7fb7c40c3060 2023-07-06T02:23:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.030+0000 7fb7c0ff9700 1 -- 172.21.15.89:0/1832647775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8003680 con 0x7fb7c40c3060 2023-07-06T02:23:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.030+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c41e2cd0 con 0x7fb7c40c3060 2023-07-06T02:23:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.031+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b0005320 con 0x7fb7c40c3060 2023-07-06T02:23:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.033+0000 7fb7c0ff9700 1 -- 172.21.15.89:0/1832647775 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7b8022030 con 0x7fb7c40c3060 2023-07-06T02:23:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.034+0000 7fb7c0ff9700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7ac06c940 0x7fb7ac06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.034+0000 7fb7c0ff9700 1 -- 172.21.15.89:0/1832647775 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb7b808aa70 con 0x7fb7c40c3060 2023-07-06T02:23:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.034+0000 7fb7c37fe700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7ac06c940 0x7fb7ac06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.035+0000 7fb7c37fe700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7ac06c940 0x7fb7ac06edf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7c41e53d0 tx=0x7fb7b4007a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.042+0000 7fb7c0ff9700 1 -- 172.21.15.89:0/1832647775 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb7b80559a0 con 0x7fb7c40c3060 2023-07-06T02:23:16.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.061+0000 7fec0009d700 1 Processor -- start 2023-07-06T02:23:16.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.061+0000 7fec0009d700 1 -- start start 2023-07-06T02:23:16.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.062+0000 7fec0009d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7febf8070580 0x7febf8070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.062+0000 7fec0009d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf8070e90 con 0x7febf80ce2e0 2023-07-06T02:23:16.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.062+0000 7fec0009d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf80cc5e0 con 0x7febf8166ac0 2023-07-06T02:23:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.062+0000 7fec0009d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf80cc8c0 con 0x7febf8070580 2023-07-06T02:23:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.063+0000 7febfe63a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7febf8070580 0x7febf8070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.063+0000 7febfde39700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7febf8166ac0 0x7febf816a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33512/0 (socket says 172.21.15.89:33512) 2023-07-06T02:23:16.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.063+0000 7febfde39700 1 -- 172.21.15.89:0/1899877315 learned_addr learned my addr 172.21.15.89:0/1899877315 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.064+0000 7febfce37700 1 -- 172.21.15.89:0/1899877315 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841702017 0 0) 0x7febf80cc5e0 con 0x7febf8166ac0 2023-07-06T02:23:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.064+0000 7febfce37700 1 -- 172.21.15.89:0/1899877315 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febe40035b0 con 0x7febf8166ac0 2023-07-06T02:23:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.065+0000 7febfe63a700 1 -- 172.21.15.89:0/1899877315 >> v1:172.21.15.172:6789/0 conn(0x7febf8166ac0 legacy=0x7febf816a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.065+0000 7febfe63a700 1 -- 172.21.15.89:0/1899877315 >> v1:172.21.15.89:6789/0 conn(0x7febf80ce2e0 legacy=0x7febf816d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:16.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.065+0000 7febfe63a700 1 -- 172.21.15.89:0/1899877315 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf816ccd0 con 0x7febf8070580 2023-07-06T02:23:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.065+0000 7febfe63a700 1 --2- 172.21.15.89:0/1899877315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8070580 0x7febf8070950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febf4007360 tx=0x7febf4009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=effd4e5c7e55c48b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.065+0000 7febfce37700 1 -- 172.21.15.89:0/1899877315 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf400e070 con 0x7febf8070580 2023-07-06T02:23:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.065+0000 7febfce37700 1 -- 172.21.15.89:0/1899877315 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf40092e0 con 0x7febf8070580 2023-07-06T02:23:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.065+0000 7febfce37700 1 -- 172.21.15.89:0/1899877315 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4004230 con 0x7febf8070580 2023-07-06T02:23:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.066+0000 7fec0009d700 1 -- 172.21.15.89:0/1899877315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8070580 msgr2=0x7febf8070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.066+0000 7fec0009d700 1 --2- 172.21.15.89:0/1899877315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8070580 0x7febf8070950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febf4007360 tx=0x7febf4009c60 comp rx=0 tx=0).stop 2023-07-06T02:23:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.066+0000 7fec0009d700 1 -- 172.21.15.89:0/1899877315 shutdown_connections 2023-07-06T02:23:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.066+0000 7fec0009d700 1 --2- 172.21.15.89:0/1899877315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8070580 0x7febf8070950 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.066+0000 7fec0009d700 1 -- 172.21.15.89:0/1899877315 >> 172.21.15.89:0/1899877315 conn(0x7febf80ce870 msgr2=0x7febf80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.067+0000 7fec0009d700 1 -- 172.21.15.89:0/1899877315 shutdown_connections 2023-07-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.067+0000 7fec0009d700 1 -- 172.21.15.89:0/1899877315 wait complete. 2023-07-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.068+0000 7fec0009d700 1 Processor -- start 2023-07-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.069+0000 7fec0009d700 1 -- start start 2023-07-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7fec0009d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febf8070580 0x7febf8084400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7fec0009d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febf80ce2e0 0x7febf8167460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7fec0009d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8166ac0 0x7febf8165840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7fec0009d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8083660 con 0x7febf80ce2e0 2023-07-06T02:23:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7fec0009d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81fffe0 con 0x7febf8166ac0 2023-07-06T02:23:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7fec0009d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf8083920 con 0x7febf8070580 2023-07-06T02:23:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfd638700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febf80ce2e0 0x7febf8167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfd638700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febf80ce2e0 0x7febf8167460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32856/0 (socket says 172.21.15.89:32856) 2023-07-06T02:23:16.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfd638700 1 -- 172.21.15.89:0/952055443 learned_addr learned my addr 172.21.15.89:0/952055443 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfde39700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febf8070580 0x7febf8084400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfe63a700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8166ac0 0x7febf8165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfde39700 1 -- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8166ac0 msgr2=0x7febf8165840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfde39700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8166ac0 0x7febf8165840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfde39700 1 -- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febf80ce2e0 msgr2=0x7febf8167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfde39700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febf80ce2e0 0x7febf8167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.070+0000 7febfde39700 1 -- 172.21.15.89:0/952055443 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4007040 con 0x7febf8070580 2023-07-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.071+0000 7febfde39700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febf8070580 0x7febf8084400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febe8001ea0 tx=0x7febe800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.073+0000 7febee7fc700 1 -- 172.21.15.89:0/952055443 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8005d70 con 0x7febf8070580 2023-07-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.073+0000 7febee7fc700 1 -- 172.21.15.89:0/952055443 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe80070d0 con 0x7febf8070580 2023-07-06T02:23:16.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.073+0000 7febee7fc700 1 -- 172.21.15.89:0/952055443 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8007440 con 0x7febf8070580 2023-07-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.073+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf8083c00 con 0x7febf8070580 2023-07-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.073+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf81679a0 con 0x7febf8070580 2023-07-06T02:23:16.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.075+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf80712e0 con 0x7febf8070580 2023-07-06T02:23:16.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.076+0000 7febee7fc700 1 -- 172.21.15.89:0/952055443 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febe8017020 con 0x7febf8070580 2023-07-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.076+0000 7febee7fc700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febdc06adb0 0x7febdc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.077+0000 7febee7fc700 1 -- 172.21.15.89:0/952055443 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7febe808cc00 con 0x7febf8070580 2023-07-06T02:23:16.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.077+0000 7febfd638700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febdc06adb0 0x7febdc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.078+0000 7febfd638700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febdc06adb0 0x7febdc06d260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7febf806df20 tx=0x7febf0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.077+0000 7f76aad48700 1 Processor -- start 2023-07-06T02:23:16.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.077+0000 7f76aad48700 1 -- start start 2023-07-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.078+0000 7f76aad48700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a415c480 0x7f76a40cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.078+0000 7f76aad48700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a415c900 con 0x7f76a415bfe0 2023-07-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.078+0000 7f76aad48700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a406ab20 con 0x7f76a4152a60 2023-07-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.078+0000 7f76aad48700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a406ae00 con 0x7f76a415c480 2023-07-06T02:23:16.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.079+0000 7f76a3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f76a415bfe0 0x7f76a40cd080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57770/0 (socket says 172.21.15.89:57770) 2023-07-06T02:23:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.079+0000 7f76a3fff700 1 -- 172.21.15.89:0/1648413749 learned_addr learned my addr 172.21.15.89:0/1648413749 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:16.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7fef0cad5700 1 Processor -- start 2023-07-06T02:23:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032477642 0 0) 0x7f76a415c900 con 0x7f76a415bfe0 2023-07-06T02:23:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f768c0035b0 con 0x7f76a415bfe0 2023-07-06T02:23:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961314924 0 0) 0x7f768c0035b0 con 0x7f76a415bfe0 2023-07-06T02:23:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a415c480 msgr2=0x7f76a40cd790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7f76a37fe700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a415c480 0x7f76a40cd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 >> v1:172.21.15.172:6789/0 conn(0x7f76a4152a60 legacy=0x7f76a406a060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a406b0e0 con 0x7f76a415bfe0 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f7698001670 con 0x7f76a415bfe0 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 >> v1:172.21.15.89:6789/0 conn(0x7f76a415bfe0 legacy=0x7f76a40cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768c003a30 0x7f768c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f768c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c005400 0x7f768c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a406ae00 con 0x7f768c003a30 2023-07-06T02:23:16.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c00c250 con 0x7f768c005400 2023-07-06T02:23:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c00c410 con 0x7f768c004850 2023-07-06T02:23:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.080+0000 7fef0cad5700 1 -- start start 2023-07-06T02:23:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7fef0cad5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fef08160a10 0x7fef0806aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7fef0cad5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef080ce360 con 0x7fef08070b30 2023-07-06T02:23:16.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7fef0cad5700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0806afb0 con 0x7fef0806b3d0 2023-07-06T02:23:16.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7fef0cad5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0806c440 con 0x7fef08160a10 2023-07-06T02:23:16.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7fef07fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fef08070b30 0x7fef0806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57776/0 (socket says 172.21.15.89:57776) 2023-07-06T02:23:16.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7fef07fff700 1 -- 172.21.15.89:0/2555269830 learned_addr learned my addr 172.21.15.89:0/2555269830 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:16.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.081+0000 7febee7fc700 1 -- 172.21.15.89:0/952055443 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7febe80579a0 con 0x7febf8070580 2023-07-06T02:23:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903832523 0 0) 0x7fef080ce360 con 0x7fef08070b30 2023-07-06T02:23:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feef00035b0 con 0x7fef08070b30 2023-07-06T02:23:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 501333532 0 0) 0x7feef00035b0 con 0x7fef08070b30 2023-07-06T02:23:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fef08160a10 msgr2=0x7fef0806aa70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:16.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fef08160a10 0x7fef0806aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 >> v1:172.21.15.172:6789/0 conn(0x7fef0806b3d0 legacy=0x7fef08070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080cc2f0 con 0x7fef08070b30 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7feefc001670 con 0x7fef08070b30 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 >> v1:172.21.15.89:6789/0 conn(0x7fef08070b30 legacy=0x7fef0806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0003a30 0x7feef0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef0004850 0x7feef0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feef0005320 0x7feef000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0806c440 con 0x7feef0004850 2023-07-06T02:23:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef000c1e0 con 0x7feef0005320 2023-07-06T02:23:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.083+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef000c3a0 con 0x7feef0003a30 2023-07-06T02:23:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f768c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a8ae4700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768c003a30 0x7f768c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a92e5700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c005400 0x7f768c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7fef06ffd700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef0004850 0x7feef0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a3fff700 1 -- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c005400 msgr2=0x7f768c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:16.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c005400 0x7f768c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a3fff700 1 -- 172.21.15.89:0/1648413749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768c003a30 msgr2=0x7f768c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768c003a30 0x7f768c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.086+0000 7f76a3fff700 1 -- 172.21.15.89:0/1648413749 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0035b0 con 0x7f768c004850 2023-07-06T02:23:16.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f768c004cc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7698004d60 tx=0x7f768c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318f2b12966fefe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef06ffd700 1 -- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0003a30 msgr2=0x7feef0003e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T02:23:16.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef06ffd700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0003a30 0x7feef0003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef06ffd700 1 -- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feef0005320 msgr2=0x7feef000ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T02:23:16.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef06ffd700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feef0005320 0x7feef000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef06ffd700 1 -- 172.21.15.89:0/2555269830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080cc2f0 con 0x7feef0004850 2023-07-06T02:23:16.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef06ffd700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef0004850 0x7feef0004cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feef8001600 tx=0x7feef0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a06962be4eeb37 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80052d0 con 0x7feef0004850 2023-07-06T02:23:16.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef8005430 con 0x7feef0004850 2023-07-06T02:23:16.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.087+0000 7fef067fc700 1 -- 172.21.15.89:0/2555269830 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80038a0 con 0x7feef0004850 2023-07-06T02:23:16.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.088+0000 7fef0cad5700 1 -- 172.21.15.89:0/2555269830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef0004850 msgr2=0x7feef0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.088+0000 7fef0cad5700 1 --2- 172.21.15.89:0/2555269830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef0004850 0x7feef0004cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feef8001600 tx=0x7feef0005880 comp rx=0 tx=0).stop 2023-07-06T02:23:16.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.089+0000 7fef07fff700 1 -- 172.21.15.89:0/2555269830 reap_dead start 2023-07-06T02:23:16.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.090+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800b490 con 0x7f768c004850 2023-07-06T02:23:16.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.090+0000 7f66b77d6700 1 Processor -- start 2023-07-06T02:23:16.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.090+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769800bd60 con 0x7f768c004850 2023-07-06T02:23:16.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.090+0000 7f76a37fe700 1 -- 172.21.15.89:0/1648413749 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800f640 con 0x7f768c004850 2023-07-06T02:23:16.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.091+0000 7f76aad48700 1 -- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 msgr2=0x7f768c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.091+0000 7f76aad48700 1 --2- 172.21.15.89:0/1648413749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f768c004cc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7698004d60 tx=0x7f768c005880 comp rx=0 tx=0).stop 2023-07-06T02:23:16.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.091+0000 7f76a92e5700 1 -- 172.21.15.89:0/1648413749 reap_dead start 2023-07-06T02:23:16.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.090+0000 7f66b77d6700 1 -- start start 2023-07-06T02:23:16.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b77d6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66b0070f20 0x7f66b00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b77d6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b00c9fa0 con 0x7f66b00712f0 2023-07-06T02:23:16.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b77d6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b006f290 con 0x7f66b006b190 2023-07-06T02:23:16.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b77d6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b006ecd0 con 0x7f66b0070f20 2023-07-06T02:23:16.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b5572700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66b0070f20 0x7f66b00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b5572700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66b0070f20 0x7f66b00c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32886/0 (socket says 172.21.15.89:32886) 2023-07-06T02:23:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b5572700 1 -- 172.21.15.89:0/1908837391 learned_addr learned my addr 172.21.15.89:0/1908837391 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f76aad48700 1 -- 172.21.15.89:0/1648413749 shutdown_connections 2023-07-06T02:23:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f76aad48700 1 -- 172.21.15.89:0/1648413749 >> 172.21.15.89:0/1648413749 conn(0x7f76a40cf8d0 msgr2=0x7f76a40cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b5572700 1 -- 172.21.15.89:0/1908837391 >> v1:172.21.15.172:6789/0 conn(0x7f66b006b190 legacy=0x7f66b006b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b5572700 1 -- 172.21.15.89:0/1908837391 >> v1:172.21.15.89:6789/0 conn(0x7f66b00712f0 legacy=0x7f66b006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f66b5572700 1 -- 172.21.15.89:0/1908837391 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b006efb0 con 0x7f66b0070f20 2023-07-06T02:23:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.092+0000 7f76aad48700 1 -- 172.21.15.89:0/1648413749 shutdown_connections 2023-07-06T02:23:16.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.093+0000 7f76aad48700 1 -- 172.21.15.89:0/1648413749 wait complete. 2023-07-06T02:23:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.094+0000 7f66b5572700 1 --2- 172.21.15.89:0/1908837391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b00c9a60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00bb90 tx=0x7f66ac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893d16ef6e67880c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.094+0000 7f66a7fff700 1 -- 172.21.15.89:0/1908837391 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00dc40 con 0x7f66b0070f20 2023-07-06T02:23:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.094+0000 7f66a7fff700 1 -- 172.21.15.89:0/1908837391 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac00dda0 con 0x7f66b0070f20 2023-07-06T02:23:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.094+0000 7f66a7fff700 1 -- 172.21.15.89:0/1908837391 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac017630 con 0x7f66b0070f20 2023-07-06T02:23:16.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.094+0000 7f76aad48700 1 Processor -- start 2023-07-06T02:23:16.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7fef0cad5700 1 -- 172.21.15.89:0/2555269830 shutdown_connections 2023-07-06T02:23:16.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7fef0cad5700 1 -- 172.21.15.89:0/2555269830 >> 172.21.15.89:0/2555269830 conn(0x7fef080ce680 msgr2=0x7fef080cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f76aad48700 1 -- start start 2023-07-06T02:23:16.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76aad48700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f76a414ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76aad48700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a414e100 0x7f76a4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76aad48700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76a414e570 0x7f76a415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76aad48700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a414d060 con 0x7f76a414e570 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76aad48700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a414d340 con 0x7f76a414e100 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76aad48700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a414d620 con 0x7f768c004850 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76a92e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76a414e570 0x7f76a415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76a92e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76a414e570 0x7f76a415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32896/0 (socket says 172.21.15.89:32896) 2023-07-06T02:23:16.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76a92e5700 1 -- 172.21.15.89:0/1266559657 learned_addr learned my addr 172.21.15.89:0/1266559657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76a8ae4700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f76a414ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.096+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a414e100 0x7f76a4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f66b77d6700 1 -- 172.21.15.89:0/1908837391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 msgr2=0x7f66b00c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f66b77d6700 1 --2- 172.21.15.89:0/1908837391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b00c9a60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00bb90 tx=0x7f66ac00bea0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f66b77d6700 1 -- 172.21.15.89:0/1908837391 shutdown_connections 2023-07-06T02:23:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f66b77d6700 1 --2- 172.21.15.89:0/1908837391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b00c9a60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f66b77d6700 1 -- 172.21.15.89:0/1908837391 >> 172.21.15.89:0/1908837391 conn(0x7f66b00cf1c0 msgr2=0x7f66b00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f66b77d6700 1 -- 172.21.15.89:0/1908837391 shutdown_connections 2023-07-06T02:23:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.095+0000 7f66b77d6700 1 -- 172.21.15.89:0/1908837391 wait complete. 2023-07-06T02:23:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f76a3fff700 1 -- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 msgr2=0x7f76a414ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f76a414ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f76a3fff700 1 -- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76a414e570 msgr2=0x7f76a415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76a414e570 0x7f76a415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f76a3fff700 1 -- 172.21.15.89:0/1266559657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a414d900 con 0x7f76a414e100 2023-07-06T02:23:16.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f66b77d6700 1 Processor -- start 2023-07-06T02:23:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f76a3fff700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a414e100 0x7f76a4159b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7698016000 tx=0x7f7698001930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f76a17fa700 1 -- 172.21.15.89:0/1266559657 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800b490 con 0x7f76a414e100 2023-07-06T02:23:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f76aad48700 1 -- 172.21.15.89:0/1266559657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a414dbe0 con 0x7f76a414e100 2023-07-06T02:23:16.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f76aad48700 1 -- 172.21.15.89:0/1266559657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a41e1eb0 con 0x7f76a414e100 2023-07-06T02:23:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.097+0000 7f66b77d6700 1 -- start start 2023-07-06T02:23:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f66b77d6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66b006b190 0x7f66b00c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f66b77d6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b01fe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f66b77d6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66b00712f0 0x7f66b00cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f66b77d6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00c89a0 con 0x7f66b0070f20 2023-07-06T02:23:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f66b77d6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b0156ec0 con 0x7f66b00712f0 2023-07-06T02:23:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.098+0000 7f66b77d6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b01571a0 con 0x7f66b006b190 2023-07-06T02:23:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b4d71700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b01fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b4d71700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b01fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32900/0 (socket says 172.21.15.89:32900) 2023-07-06T02:23:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b4d71700 1 -- 172.21.15.89:0/619395271 learned_addr learned my addr 172.21.15.89:0/619395271 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5572700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66b006b190 0x7f66b00c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5d73700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66b00712f0 0x7f66b00cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5572700 1 -- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66b00712f0 msgr2=0x7f66b00cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5572700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66b00712f0 0x7f66b00cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5572700 1 -- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 msgr2=0x7f66b01fe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5572700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b01fe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5572700 1 -- 172.21.15.89:0/619395271 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac00b7f0 con 0x7f66b006b190 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b4d71700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b01fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:16.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.099+0000 7f66b5572700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66b006b190 0x7f66b00c8460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66ac018000 tx=0x7f66ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.100+0000 7f66a5ffb700 1 -- 172.21.15.89:0/619395271 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0038b0 con 0x7f66b006b190 2023-07-06T02:23:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.100+0000 7f66a5ffb700 1 -- 172.21.15.89:0/619395271 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac003a70 con 0x7f66b006b190 2023-07-06T02:23:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.100+0000 7f66b77d6700 1 -- 172.21.15.89:0/619395271 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b0157480 con 0x7f66b006b190 2023-07-06T02:23:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.100+0000 7f66a5ffb700 1 -- 172.21.15.89:0/619395271 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac003e20 con 0x7f66b006b190 2023-07-06T02:23:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.100+0000 7f66b77d6700 1 -- 172.21.15.89:0/619395271 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b01feac0 con 0x7f66b006b190 2023-07-06T02:23:16.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.101+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c0052f0 con 0x7f66b006b190 2023-07-06T02:23:16.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.102+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a4074270 con 0x7f76a414e100 2023-07-06T02:23:16.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.103+0000 7f66a5ffb700 1 -- 172.21.15.89:0/619395271 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66ac015070 con 0x7f66b006b190 2023-07-06T02:23:16.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.104+0000 7f66a5ffb700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f669406ad30 0x7f669406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.104+0000 7f66a5ffb700 1 -- 172.21.15.89:0/619395271 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f66ac08bc10 con 0x7f66b006b190 2023-07-06T02:23:16.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.107+0000 7fef0cad5700 1 -- 172.21.15.89:0/2555269830 shutdown_connections 2023-07-06T02:23:16.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.107+0000 7f66a5ffb700 1 -- 172.21.15.89:0/619395271 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f66ac056b40 con 0x7f66b006b190 2023-07-06T02:23:16.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.107+0000 7fef0cad5700 1 -- 172.21.15.89:0/2555269830 wait complete. 2023-07-06T02:23:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.109+0000 7f66b4d71700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f669406ad30 0x7f669406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.109+0000 7fef0cad5700 1 Processor -- start 2023-07-06T02:23:16.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.109+0000 7f66b4d71700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f669406ad30 0x7f669406d1e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66b00c9b10 tx=0x7f66a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.109+0000 7fef0cad5700 1 -- start start 2023-07-06T02:23:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef0cad5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0004850 0x7fef081fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef0cad5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef080adae0 0x7fef0815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef0cad5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef081f6280 0x7fef080b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef0cad5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef081f9dd0 con 0x7fef080adae0 2023-07-06T02:23:16.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef0cad5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef081fa710 con 0x7fef081f6280 2023-07-06T02:23:16.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef0cad5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0815c0a0 con 0x7feef0004850 2023-07-06T02:23:16.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef07fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef081f6280 0x7fef080b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef07fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef081f6280 0x7fef080b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59896/0 (socket says 172.21.15.89:59896) 2023-07-06T02:23:16.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef07fff700 1 -- 172.21.15.89:0/1891373123 learned_addr learned my addr 172.21.15.89:0/1891373123 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:16.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef077fe700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0004850 0x7fef081fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef06ffd700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef080adae0 0x7fef0815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef077fe700 1 -- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef081f6280 msgr2=0x7fef080b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef077fe700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef081f6280 0x7fef080b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef077fe700 1 -- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef080adae0 msgr2=0x7fef0815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef077fe700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef080adae0 0x7fef0815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.111+0000 7fef077fe700 1 -- 172.21.15.89:0/1891373123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef081f6c60 con 0x7feef0004850 2023-07-06T02:23:16.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.112+0000 7fef077fe700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0004850 0x7fef081fa1d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef0806be80 tx=0x7fef0000bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.116+0000 7feeeffff700 1 -- 172.21.15.89:0/1891373123 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000d860 con 0x7feef0004850 2023-07-06T02:23:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7feeeffff700 1 -- 172.21.15.89:0/1891373123 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0000d9c0 con 0x7feef0004850 2023-07-06T02:23:16.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7feeeffff700 1 -- 172.21.15.89:0/1891373123 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000dd70 con 0x7feef0004850 2023-07-06T02:23:16.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7f76a17fa700 1 -- 172.21.15.89:0/1266559657 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76980019e0 con 0x7f76a414e100 2023-07-06T02:23:16.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7f76a17fa700 1 -- 172.21.15.89:0/1266559657 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698027420 con 0x7f76a414e100 2023-07-06T02:23:16.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7f76a17fa700 1 -- 172.21.15.89:0/1266559657 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f769800a070 con 0x7f76a414e100 2023-07-06T02:23:16.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7f76a17fa700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f768806ad30 0x7f768806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7f76a17fa700 1 -- 172.21.15.89:0/1266559657 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7698006070 con 0x7f76a414e100 2023-07-06T02:23:16.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.117+0000 7f76a17fa700 1 -- 172.21.15.89:0/1266559657 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f769808f2a0 con 0x7f76a414e100 2023-07-06T02:23:16.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.118+0000 7f76a8ae4700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f768806ad30 0x7f768806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.118+0000 7f76a8ae4700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f768806ad30 0x7f768806d1e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f769400bae0 tx=0x7f7694008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.120+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef081f6f40 con 0x7feef0004850 2023-07-06T02:23:16.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.120+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef081f66f0 con 0x7feef0004850 2023-07-06T02:23:16.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.128+0000 7feeeffff700 1 -- 172.21.15.89:0/1891373123 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef00017400 con 0x7feef0004850 2023-07-06T02:23:16.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.128+0000 7feeeffff700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feef406adb0 0x7feef406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:16.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.128+0000 7feeeffff700 1 -- 172.21.15.89:0/1891373123 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fef000896e0 con 0x7feef0004850 2023-07-06T02:23:16.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.129+0000 7fef06ffd700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feef406adb0 0x7feef406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:16.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.129+0000 7fef06ffd700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feef406adb0 0x7feef406d260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feef8000c00 tx=0x7feef8010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:16.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.135+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef081fa040 con 0x7feef0004850 2023-07-06T02:23:16.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.142+0000 7feeeffff700 1 -- 172.21.15.89:0/1891373123 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef00054510 con 0x7feef0004850 2023-07-06T02:23:16.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.234+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3478005870 con 0x7f3498141870 2023-07-06T02:23:16.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.236+0000 7f348dffb700 1 -- 172.21.15.89:0/3173105200 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3490017030 con 0x7f3498141870 2023-07-06T02:23:16.244 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:16.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.242+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f347c06ace0 msgr2=0x7f347c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.242+0000 7f349f8c6700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f347c06ace0 0x7f347c06d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3488000c00 tx=0x7f348801c040 comp rx=0 tx=0).stop 2023-07-06T02:23:16.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.242+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3498141870 msgr2=0x7f34980c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.242+0000 7f349f8c6700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3498141870 0x7f34980c4880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f349000cb10 tx=0x7f349000ce20 comp rx=0 tx=0).stop 2023-07-06T02:23:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 shutdown_connections 2023-07-06T02:23:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f347c06ace0 0x7f347c06d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3498141870 0x7f34980c4880 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34980707f0 0x7f34981f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 --2- 172.21.15.89:0/3173105200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3498070bc0 0x7f34980c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 >> 172.21.15.89:0/3173105200 conn(0x7f34980ce870 msgr2=0x7f34980cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 shutdown_connections 2023-07-06T02:23:16.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.243+0000 7f349f8c6700 1 -- 172.21.15.89:0/3173105200 wait complete. 2023-07-06T02:23:16.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-07-06T02:23:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.393+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05880055a0 con 0x7f05a806c710 2023-07-06T02:23:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.394+0000 7f05a4ff9700 1 -- 172.21.15.89:0/3423390050 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05a001a020 con 0x7f05a806c710 2023-07-06T02:23:16.397 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.398+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f059406adb0 msgr2=0x7f059406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.398+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f059406adb0 0x7f059406d260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cc580 tx=0x7f0598008030 comp rx=0 tx=0).stop 2023-07-06T02:23:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.398+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05a806c710 msgr2=0x7f05a814e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.398+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05a806c710 0x7f05a814e010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05a00070b0 tx=0x7f05a0007f30 comp rx=0 tx=0).stop 2023-07-06T02:23:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 shutdown_connections 2023-07-06T02:23:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a80c25b0 0x7f05a8148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f059406adb0 0x7f059406d260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05a806c710 0x7f05a814e010 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 --2- 172.21.15.89:0/3423390050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a8070830 0x7f05a814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 >> 172.21.15.89:0/3423390050 conn(0x7f05a80ceaf0 msgr2=0x7f05a8160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fef081f9850 con 0x7feef0004850 2023-07-06T02:23:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 shutdown_connections 2023-07-06T02:23:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7f05ad0dc700 1 -- 172.21.15.89:0/3423390050 wait complete. 2023-07-06T02:23:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.399+0000 7feeeffff700 1 -- 172.21.15.89:0/1891373123 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef081f9850 con 0x7feef0004850 2023-07-06T02:23:16.404 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-07-06T02:23:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feef406adb0 msgr2=0x7feef406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feef406adb0 0x7feef406d260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feef8000c00 tx=0x7feef8010000 comp rx=0 tx=0).stop 2023-07-06T02:23:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0004850 msgr2=0x7fef081fa1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0004850 0x7fef081fa1d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef0806be80 tx=0x7fef0000bd60 comp rx=0 tx=0).stop 2023-07-06T02:23:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 shutdown_connections 2023-07-06T02:23:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef081f6280 0x7fef080b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feef0004850 0x7fef081fa1d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7feef406adb0 0x7feef406d260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 --2- 172.21.15.89:0/1891373123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef080adae0 0x7fef0815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 >> 172.21.15.89:0/1891373123 conn(0x7fef080ce680 msgr2=0x7fef080b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 shutdown_connections 2023-07-06T02:23:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.406+0000 7fef0cad5700 1 -- 172.21.15.89:0/1891373123 wait complete. 2023-07-06T02:23:16.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-07-06T02:23:16.417 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.415+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f19a0005870 con 0x7f19b8141870 2023-07-06T02:23:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.416+0000 7f19ae7fc700 1 -- 172.21.15.89:0/3461238891 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f19b0059970 con 0x7f19b8141870 2023-07-06T02:23:16.421 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.419+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f19a406ae40 msgr2=0x7f19a406d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.419+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f19a406ae40 0x7f19a406d2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009770 tx=0x7f19a8006cd0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.419+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b8141870 msgr2=0x7f19b80c1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.419+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b8141870 0x7f19b80c1360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d6c0 tx=0x7f19b0004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 shutdown_connections 2023-07-06T02:23:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f19a406ae40 0x7f19a406d2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19b8141870 0x7f19b80c1360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19b806dcb0 0x7f19b80ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 --2- 172.21.15.89:0/3461238891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19b80707f0 0x7f19b80c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 >> 172.21.15.89:0/3461238891 conn(0x7f19b80ce870 msgr2=0x7f19b80af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 shutdown_connections 2023-07-06T02:23:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.420+0000 7f19bfdd5700 1 -- 172.21.15.89:0/3461238891 wait complete. 2023-07-06T02:23:16.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-07-06T02:23:16.429 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:16.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-07-06T02:23:16.437 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:16.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.443+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb7b00058a0 con 0x7fb7c40c3060 2023-07-06T02:23:16.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.443+0000 7fb7c0ff9700 1 -- 172.21.15.89:0/1832647775 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7b8058fc0 con 0x7fb7c40c3060 2023-07-06T02:23:16.446 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.445+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7ac06c940 msgr2=0x7fb7ac06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.445+0000 7fb7ca4c8700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7ac06c940 0x7fb7ac06edf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7c41e53d0 tx=0x7fb7b4007a50 comp rx=0 tx=0).stop 2023-07-06T02:23:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.445+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40c3060 msgr2=0x7fb7c40ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.445+0000 7fb7ca4c8700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40c3060 0x7fb7c40ca120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8014040 tx=0x7fb7b8001a00 comp rx=0 tx=0).stop 2023-07-06T02:23:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 shutdown_connections 2023-07-06T02:23:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c40c3970 0x7fb7c41e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7ac06c940 0x7fb7ac06edf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40c3060 0x7fb7c40ca120 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 --2- 172.21.15.89:0/1832647775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c40ca660 0x7fb7c40c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 >> 172.21.15.89:0/1832647775 conn(0x7fb7c40ce870 msgr2=0x7fb7c40b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 shutdown_connections 2023-07-06T02:23:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.446+0000 7fb7ca4c8700 1 -- 172.21.15.89:0/1832647775 wait complete. 2023-07-06T02:23:16.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-07-06T02:23:16.461 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.467+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7febf8084120 con 0x7febf8070580 2023-07-06T02:23:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.468+0000 7febee7fc700 1 -- 172.21.15.89:0/952055443 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febf8084120 con 0x7febf8070580 2023-07-06T02:23:16.471 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.468+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f668c005870 con 0x7f66b006b190 2023-07-06T02:23:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.469+0000 7f66a5ffb700 1 -- 172.21.15.89:0/619395271 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66ac05a160 con 0x7f66b006b190 2023-07-06T02:23:16.472 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-07-06T02:23:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febdc06adb0 msgr2=0x7febdc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febdc06adb0 0x7febdc06d260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7febf806df20 tx=0x7febf0009040 comp rx=0 tx=0).stop 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febf8070580 msgr2=0x7febf8084400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febf8070580 0x7febf8084400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febe8001ea0 tx=0x7febe800c710 comp rx=0 tx=0).stop 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 shutdown_connections 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febf8166ac0 0x7febf8165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febf8070580 0x7febf8084400 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7febdc06adb0 0x7febdc06d260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 --2- 172.21.15.89:0/952055443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febf80ce2e0 0x7febf8167460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 >> 172.21.15.89:0/952055443 conn(0x7febf80ce870 msgr2=0x7febf816b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 shutdown_connections 2023-07-06T02:23:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.470+0000 7fec0009d700 1 -- 172.21.15.89:0/952055443 wait complete. 2023-07-06T02:23:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f669406ad30 msgr2=0x7f669406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f669406ad30 0x7f669406d1e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66b00c9b10 tx=0x7f66a0009040 comp rx=0 tx=0).stop 2023-07-06T02:23:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66b006b190 msgr2=0x7f66b00c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66b006b190 0x7f66b00c8460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66ac018000 tx=0x7f66ac004750 comp rx=0 tx=0).stop 2023-07-06T02:23:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 shutdown_connections 2023-07-06T02:23:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66b00712f0 0x7f66b00cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66b006b190 0x7f66b00c8460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f669406ad30 0x7f669406d1e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 --2- 172.21.15.89:0/619395271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66b0070f20 0x7f66b01fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 >> 172.21.15.89:0/619395271 conn(0x7f66b00cf1c0 msgr2=0x7f66b00b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 shutdown_connections 2023-07-06T02:23:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.472+0000 7f66937fe700 1 -- 172.21.15.89:0/619395271 wait complete. 2023-07-06T02:23:16.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-07-06T02:23:16.486 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:16.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-07-06T02:23:16.493 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.502+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f76a414d480 con 0x7f76a414e100 2023-07-06T02:23:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.502+0000 7f76a17fa700 1 -- 172.21.15.89:0/1266559657 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f76a414d480 con 0x7f76a414e100 2023-07-06T02:23:16.505 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-07-06T02:23:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.504+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f768806ad30 msgr2=0x7f768806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.504+0000 7f7692ffd700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f768806ad30 0x7f768806d1e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f769400bae0 tx=0x7f7694008040 comp rx=0 tx=0).stop 2023-07-06T02:23:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.504+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a414e100 msgr2=0x7f76a4159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.504+0000 7f7692ffd700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a414e100 0x7f76a4159b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7698016000 tx=0x7f7698001930 comp rx=0 tx=0).stop 2023-07-06T02:23:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 shutdown_connections 2023-07-06T02:23:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f768806ad30 0x7f768806d1e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76a414e570 0x7f76a415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c004850 0x7f76a414ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 --2- 172.21.15.89:0/1266559657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a414e100 0x7f76a4159b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 >> 172.21.15.89:0/1266559657 conn(0x7f76a40cf8d0 msgr2=0x7f76a40709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 shutdown_connections 2023-07-06T02:23:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:16.505+0000 7f7692ffd700 1 -- 172.21.15.89:0/1266559657 wait complete. 2023-07-06T02:23:16.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2023-07-06T02:23:16.520 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:17.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T02:23:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.395+0000 7f4b3c4ba700 1 Processor -- start 2023-07-06T02:23:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.395+0000 7f4b3c4ba700 1 -- start start 2023-07-06T02:23:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.396+0000 7f4b3c4ba700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b340712f0 0x7f4b3406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.396+0000 7f4b3c4ba700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b3406e790 0x7f4b3406b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.396+0000 7f4b3c4ba700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3406f290 con 0x7f4b34070f20 2023-07-06T02:23:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.396+0000 7f4b3c4ba700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3406ecd0 con 0x7f4b3406e790 2023-07-06T02:23:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.396+0000 7f4b3c4ba700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3406efb0 con 0x7f4b340712f0 2023-07-06T02:23:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b3406e790 0x7f4b3406b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b3406e790 0x7f4b3406b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32918/0 (socket says 172.21.15.89:32918) 2023-07-06T02:23:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 -- 172.21.15.89:0/88647932 learned_addr learned my addr 172.21.15.89:0/88647932 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b39a55700 1 --2- 172.21.15.89:0/88647932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b340712f0 0x7f4b3406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 -- 172.21.15.89:0/88647932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b340712f0 msgr2=0x7f4b3406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 --2- 172.21.15.89:0/88647932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b340712f0 0x7f4b3406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 -- 172.21.15.89:0/88647932 >> v1:172.21.15.172:6789/0 conn(0x7f4b34070f20 legacy=0x7f4b340c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 -- 172.21.15.89:0/88647932 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3406a990 con 0x7f4b3406e790 2023-07-06T02:23:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b3aa57700 1 --2- 172.21.15.89:0/88647932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3406e790 0x7f4b3406b190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009e50 tx=0x7f4b2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6048438acfacf7f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b39254700 1 -- 172.21.15.89:0/88647932 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00e070 con 0x7f4b3406e790 2023-07-06T02:23:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b39254700 1 -- 172.21.15.89:0/88647932 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c003a20 con 0x7f4b3406e790 2023-07-06T02:23:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.397+0000 7f4b39254700 1 -- 172.21.15.89:0/88647932 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c003dd0 con 0x7f4b3406e790 2023-07-06T02:23:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/88647932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3406e790 msgr2=0x7f4b3406b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/88647932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3406e790 0x7f4b3406b190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009e50 tx=0x7f4b2c00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/88647932 shutdown_connections 2023-07-06T02:23:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/88647932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b3406e790 0x7f4b3406b190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/88647932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b340712f0 0x7f4b3406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/88647932 >> 172.21.15.89:0/88647932 conn(0x7f4b340cf1c0 msgr2=0x7f4b340cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/88647932 shutdown_connections 2023-07-06T02:23:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.398+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/88647932 wait complete. 2023-07-06T02:23:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.399+0000 7f4b3c4ba700 1 Processor -- start 2023-07-06T02:23:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.399+0000 7f4b3c4ba700 1 -- start start 2023-07-06T02:23:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3c4ba700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b3406e790 0x7f4b341f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3c4ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b34070f20 0x7f4b340cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3c4ba700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b340712f0 0x7f4b340cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3c4ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340b4e80 con 0x7f4b34070f20 2023-07-06T02:23:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3c4ba700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b341f13d0 con 0x7f4b340712f0 2023-07-06T02:23:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3c4ba700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b341eecc0 con 0x7f4b3406e790 2023-07-06T02:23:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b34070f20 0x7f4b340cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b34070f20 0x7f4b340cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32922/0 (socket says 172.21.15.89:32922) 2023-07-06T02:23:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 -- 172.21.15.89:0/3050147259 learned_addr learned my addr 172.21.15.89:0/3050147259 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:17.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3a256700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b3406e790 0x7f4b341f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3aa57700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b340712f0 0x7f4b340cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 -- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b3406e790 msgr2=0x7f4b341f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b3406e790 0x7f4b341f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 -- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b340712f0 msgr2=0x7f4b340cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b340712f0 0x7f4b340cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 -- 172.21.15.89:0/3050147259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c009b30 con 0x7f4b34070f20 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b39a55700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b34070f20 0x7f4b340cabb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400ac40 tx=0x7f4b2400af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:17.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b2affd700 1 -- 172.21.15.89:0/3050147259 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24012070 con 0x7f4b34070f20 2023-07-06T02:23:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b2affd700 1 -- 172.21.15.89:0/3050147259 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2400f600 con 0x7f4b34070f20 2023-07-06T02:23:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b2affd700 1 -- 172.21.15.89:0/3050147259 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2400f9b0 con 0x7f4b34070f20 2023-07-06T02:23:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.400+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b341eefa0 con 0x7f4b34070f20 2023-07-06T02:23:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.401+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b340c76c0 con 0x7f4b34070f20 2023-07-06T02:23:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.401+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b34066fb0 con 0x7f4b34070f20 2023-07-06T02:23:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.402+0000 7f4b2affd700 1 -- 172.21.15.89:0/3050147259 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b24016070 con 0x7f4b34070f20 2023-07-06T02:23:17.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.402+0000 7f4b2affd700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4b1806ace0 0x7f4b1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.402+0000 7f4b2affd700 1 -- 172.21.15.89:0/3050147259 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4b2408a9a0 con 0x7f4b34070f20 2023-07-06T02:23:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.402+0000 7f4b3a256700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4b1806ace0 0x7f4b1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.403+0000 7f4b3a256700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4b1806ace0 0x7f4b1806d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b340b28e0 tx=0x7f4b30007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:17.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.407+0000 7f4b2affd700 1 -- 172.21.15.89:0/3050147259 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4b240558a0 con 0x7f4b34070f20 2023-07-06T02:23:17.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.600+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4b34074270 con 0x7f4b34070f20 2023-07-06T02:23:17.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.600+0000 7f4b2affd700 1 -- 172.21.15.89:0/3050147259 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b2400b040 con 0x7f4b34070f20 2023-07-06T02:23:17.603 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-07-06T02:23:17.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4b1806ace0 msgr2=0x7f4b1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4b1806ace0 0x7f4b1806d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b340b28e0 tx=0x7f4b30007ec0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b34070f20 msgr2=0x7f4b340cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b34070f20 0x7f4b340cabb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400ac40 tx=0x7f4b2400af50 comp rx=0 tx=0).stop 2023-07-06T02:23:17.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 shutdown_connections 2023-07-06T02:23:17.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b340712f0 0x7f4b340cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b3406e790 0x7f4b341f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4b1806ace0 0x7f4b1806d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 --2- 172.21.15.89:0/3050147259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b34070f20 0x7f4b340cabb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 >> 172.21.15.89:0/3050147259 conn(0x7f4b340cf1c0 msgr2=0x7f4b340cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:17.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 shutdown_connections 2023-07-06T02:23:17.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.602+0000 7f4b3c4ba700 1 -- 172.21.15.89:0/3050147259 wait complete. 2023-07-06T02:23:17.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2023-07-06T02:23:17.618 DEBUG:teuthology.parallel:result is None 2023-07-06T02:23:17.618 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T02:23:17.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T02:23:17.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.735+0000 7f21051ea700 1 Processor -- start 2023-07-06T02:23:17.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.735+0000 7f21051ea700 1 -- start start 2023-07-06T02:23:17.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f21051ea700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2100156ab0 0x7f2100160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f21051ea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2100160bb0 0x7f21000b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f21051ea700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21000cb3d0 con 0x7f21001600b0 2023-07-06T02:23:17.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f21051ea700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2100156e80 con 0x7f2100160bb0 2023-07-06T02:23:17.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f21051ea700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21000ca370 con 0x7f2100156ab0 2023-07-06T02:23:17.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f20fe59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2100160bb0 0x7f21000b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f20fe59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2100160bb0 0x7f21000b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32928/0 (socket says 172.21.15.89:32928) 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.736+0000 7f20fe59c700 1 -- 172.21.15.89:0/835903200 learned_addr learned my addr 172.21.15.89:0/835903200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fed9d700 1 --2- 172.21.15.89:0/835903200 >> v2:172.21.15.172:3300/0 conn(0x7f2100156ab0 0x7f2100160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fe59c700 1 -- 172.21.15.89:0/835903200 >> v2:172.21.15.172:3300/0 conn(0x7f2100156ab0 msgr2=0x7f2100160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fe59c700 1 --2- 172.21.15.89:0/835903200 >> v2:172.21.15.172:3300/0 conn(0x7f2100156ab0 0x7f2100160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fe59c700 1 -- 172.21.15.89:0/835903200 >> v1:172.21.15.172:6789/0 conn(0x7f21001600b0 legacy=0x7f210006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fe59c700 1 -- 172.21.15.89:0/835903200 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21000ca610 con 0x7f2100160bb0 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fe59c700 1 --2- 172.21.15.89:0/835903200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2100160bb0 0x7f21000b1490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f210006e9e0 tx=0x7f20f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4759947f77180d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:17.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fdd9b700 1 -- 172.21.15.89:0/835903200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0004230 con 0x7f2100160bb0 2023-07-06T02:23:17.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fdd9b700 1 -- 172.21.15.89:0/835903200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f0004390 con 0x7f2100160bb0 2023-07-06T02:23:17.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f20fdd9b700 1 -- 172.21.15.89:0/835903200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00036a0 con 0x7f2100160bb0 2023-07-06T02:23:17.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 -- 172.21.15.89:0/835903200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2100160bb0 msgr2=0x7f21000b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 --2- 172.21.15.89:0/835903200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2100160bb0 0x7f21000b1490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f210006e9e0 tx=0x7f20f0009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 -- 172.21.15.89:0/835903200 shutdown_connections 2023-07-06T02:23:17.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 --2- 172.21.15.89:0/835903200 >> v2:172.21.15.172:3300/0 conn(0x7f2100156ab0 0x7f2100160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 --2- 172.21.15.89:0/835903200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2100160bb0 0x7f21000b1490 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 -- 172.21.15.89:0/835903200 >> 172.21.15.89:0/835903200 conn(0x7f21000cc490 msgr2=0x7f21000cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:17.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 -- 172.21.15.89:0/835903200 shutdown_connections 2023-07-06T02:23:17.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.737+0000 7f21051ea700 1 -- 172.21.15.89:0/835903200 wait complete. 2023-07-06T02:23:17.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.739+0000 7f21051ea700 1 Processor -- start 2023-07-06T02:23:17.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.739+0000 7f21051ea700 1 -- start start 2023-07-06T02:23:17.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2100156ab0 0x7f21000c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21001600b0 0x7f21000b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2100148ca0 0x7f210015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000c3680 con 0x7f21001600b0 2023-07-06T02:23:17.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000b2da0 con 0x7f2100148ca0 2023-07-06T02:23:17.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21000b4e40 con 0x7f2100156ab0 2023-07-06T02:23:17.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21001600b0 0x7f21000b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21001600b0 0x7f21000b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32936/0 (socket says 172.21.15.89:32936) 2023-07-06T02:23:17.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 -- 172.21.15.89:0/2681452568 learned_addr learned my addr 172.21.15.89:0/2681452568 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:17.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fed9d700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2100156ab0 0x7f21000c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20ff59e700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2100148ca0 0x7f210015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 -- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2100156ab0 msgr2=0x7f21000c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2100156ab0 0x7f21000c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 -- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2100148ca0 msgr2=0x7f210015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2100148ca0 0x7f210015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 -- 172.21.15.89:0/2681452568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f0009550 con 0x7f21001600b0 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20ff59e700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2100148ca0 0x7f210015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:17.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20fe59c700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21001600b0 0x7f21000b2860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009130 tx=0x7f20f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20e77fe700 1 -- 172.21.15.89:0/2681452568 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0003d10 con 0x7f21001600b0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20e77fe700 1 -- 172.21.15.89:0/2681452568 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f001f070 con 0x7f21001600b0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21000b5120 con 0x7f21001600b0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f20e77fe700 1 -- 172.21.15.89:0/2681452568 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0011870 con 0x7f21001600b0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.740+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21000b56e0 con 0x7f21001600b0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.741+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21000c3830 con 0x7f21001600b0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.742+0000 7f20e77fe700 1 -- 172.21.15.89:0/2681452568 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20f0021030 con 0x7f21001600b0 2023-07-06T02:23:17.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.742+0000 7f20e77fe700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20ec06ace0 0x7f20ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:17.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.742+0000 7f20e77fe700 1 -- 172.21.15.89:0/2681452568 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f20f008ad10 con 0x7f21001600b0 2023-07-06T02:23:17.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.742+0000 7f20fed9d700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20ec06ace0 0x7f20ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:17.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.743+0000 7f20fed9d700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20ec06ace0 0x7f20ec06d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006fd0 tx=0x7f20f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:17.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.747+0000 7f20e77fe700 1 -- 172.21.15.89:0/2681452568 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21000c3830 con 0x7f21001600b0 2023-07-06T02:23:17.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.930+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 --> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21000ca8f0 con 0x7f20ec06ace0 2023-07-06T02:23:17.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.939+0000 7f20e77fe700 1 -- 172.21.15.89:0/2681452568 <== mgr.4100 v2:172.21.15.89:6832/3591115711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367567 (secure 0 0 0) 0x7f20f400c040 con 0x7f20ec06ace0 2023-07-06T02:23:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:17.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20ec06ace0 msgr2=0x7f20ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20ec06ace0 0x7f20ec06d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006fd0 tx=0x7f20f4008040 comp rx=0 tx=0).stop 2023-07-06T02:23:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21001600b0 msgr2=0x7f21000b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21001600b0 0x7f21000b2860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009130 tx=0x7f20f0004750 comp rx=0 tx=0).stop 2023-07-06T02:23:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 shutdown_connections 2023-07-06T02:23:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2100148ca0 0x7f210015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2100156ab0 0x7f21000c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f20ec06ace0 0x7f20ec06d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 --2- 172.21.15.89:0/2681452568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21001600b0 0x7f21000b2860 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:17.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 >> 172.21.15.89:0/2681452568 conn(0x7f21000cc490 msgr2=0x7f210015e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:17.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 shutdown_connections 2023-07-06T02:23:17.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:17.943+0000 7f21051ea700 1 -- 172.21.15.89:0/2681452568 wait complete. 2023-07-06T02:23:17.949 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-07-06T02:23:17.958 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-07-06T02:23:15.985452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218820,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755900,"statfs":{"total":773094113280,"available":772870041600,"internally_reserved":0,"allocated":3280896,"data_stored":2026399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005682"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462978+0000","last_change":"2023-07-06T02:22:53.433054+0000","last_active":"2023-07-06T02:22:58.462978+0000","last_peered":"2023-07-06T02:22:58.462978+0000","last_clean":"2023-07-06T02:22:58.462978+0000","last_became_active":"2023-07-06T02:22:53.432749+0000","last_became_peered":"2023-07-06T02:22:53.432749+0000","last_unstale":"2023-07-06T02:22:58.462978+0000","last_undegraded":"2023-07-06T02:22:58.462978+0000","last_fullsized":"2023-07-06T02:22:58.462978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:42.597214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643097+0000","last_change":"2023-07-06T02:22:53.425731+0000","last_active":"2023-07-06T02:22:59.643097+0000","last_peered":"2023-07-06T02:22:59.643097+0000","last_clean":"2023-07-06T02:22:59.643097+0000","last_became_active":"2023-07-06T02:22:53.425403+0000","last_became_peered":"2023-07-06T02:22:53.425403+0000","last_unstale":"2023-07-06T02:22:59.643097+0000","last_undegraded":"2023-07-06T02:22:59.643097+0000","last_fullsized":"2023-07-06T02:22:59.643097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:00.409252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623835+0000","last_change":"2023-07-06T02:22:53.413131+0000","last_active":"2023-07-06T02:22:58.623835+0000","last_peered":"2023-07-06T02:22:58.623835+0000","last_clean":"2023-07-06T02:22:58.623835+0000","last_became_active":"2023-07-06T02:22:53.412839+0000","last_became_peered":"2023-07-06T02:22:53.412839+0000","last_unstale":"2023-07-06T02:22:58.623835+0000","last_undegraded":"2023-07-06T02:22:58.623835+0000","last_fullsized":"2023-07-06T02:22:58.623835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:51.080641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463166+0000","last_change":"2023-07-06T02:22:53.420056+0000","last_active":"2023-07-06T02:22:58.463166+0000","last_peered":"2023-07-06T02:22:58.463166+0000","last_clean":"2023-07-06T02:22:58.463166+0000","last_became_active":"2023-07-06T02:22:53.419779+0000","last_became_peered":"2023-07-06T02:22:53.419779+0000","last_unstale":"2023-07-06T02:22:58.463166+0000","last_undegraded":"2023-07-06T02:22:58.463166+0000","last_fullsized":"2023-07-06T02:22:58.463166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:37.102968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642705+0000","last_change":"2023-07-06T02:22:53.420867+0000","last_active":"2023-07-06T02:22:59.642705+0000","last_peered":"2023-07-06T02:22:59.642705+0000","last_clean":"2023-07-06T02:22:59.642705+0000","last_became_active":"2023-07-06T02:22:53.419716+0000","last_became_peered":"2023-07-06T02:22:53.419716+0000","last_unstale":"2023-07-06T02:22:59.642705+0000","last_undegraded":"2023-07-06T02:22:59.642705+0000","last_fullsized":"2023-07-06T02:22:59.642705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:33.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623263+0000","last_change":"2023-07-06T02:22:53.412521+0000","last_active":"2023-07-06T02:22:58.623263+0000","last_peered":"2023-07-06T02:22:58.623263+0000","last_clean":"2023-07-06T02:22:58.623263+0000","last_became_active":"2023-07-06T02:22:53.412230+0000","last_became_peered":"2023-07-06T02:22:53.412230+0000","last_unstale":"2023-07-06T02:22:58.623263+0000","last_undegraded":"2023-07-06T02:22:58.623263+0000","last_fullsized":"2023-07-06T02:22:58.623263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:44.353524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545634+0000","last_change":"2023-07-06T02:22:53.427954+0000","last_active":"2023-07-06T02:22:58.545634+0000","last_peered":"2023-07-06T02:22:58.545634+0000","last_clean":"2023-07-06T02:22:58.545634+0000","last_became_active":"2023-07-06T02:22:53.426315+0000","last_became_peered":"2023-07-06T02:22:53.426315+0000","last_unstale":"2023-07-06T02:22:58.545634+0000","last_undegraded":"2023-07-06T02:22:58.545634+0000","last_fullsized":"2023-07-06T02:22:58.545634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642615+0000","last_change":"2023-07-06T02:22:53.425882+0000","last_active":"2023-07-06T02:22:59.642615+0000","last_peered":"2023-07-06T02:22:59.642615+0000","last_clean":"2023-07-06T02:22:59.642615+0000","last_became_active":"2023-07-06T02:22:53.425583+0000","last_became_peered":"2023-07-06T02:22:53.425583+0000","last_unstale":"2023-07-06T02:22:59.642615+0000","last_undegraded":"2023-07-06T02:22:59.642615+0000","last_fullsized":"2023-07-06T02:22:59.642615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:51.387481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642268+0000","last_change":"2023-07-06T02:22:53.427106+0000","last_active":"2023-07-06T02:22:59.642268+0000","last_peered":"2023-07-06T02:22:59.642268+0000","last_clean":"2023-07-06T02:22:59.642268+0000","last_became_active":"2023-07-06T02:22:53.426852+0000","last_became_peered":"2023-07-06T02:22:53.426852+0000","last_unstale":"2023-07-06T02:22:59.642268+0000","last_undegraded":"2023-07-06T02:22:59.642268+0000","last_fullsized":"2023-07-06T02:22:59.642268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643507+0000","last_change":"2023-07-06T02:22:53.421592+0000","last_active":"2023-07-06T02:22:59.643507+0000","last_peered":"2023-07-06T02:22:59.643507+0000","last_clean":"2023-07-06T02:22:59.643507+0000","last_became_active":"2023-07-06T02:22:53.421245+0000","last_became_peered":"2023-07-06T02:22:53.421245+0000","last_unstale":"2023-07-06T02:22:59.643507+0000","last_undegraded":"2023-07-06T02:22:59.643507+0000","last_fullsized":"2023-07-06T02:22:59.643507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623746+0000","last_change":"2023-07-06T02:22:53.424792+0000","last_active":"2023-07-06T02:22:58.623746+0000","last_peered":"2023-07-06T02:22:58.623746+0000","last_clean":"2023-07-06T02:22:58.623746+0000","last_became_active":"2023-07-06T02:22:53.424498+0000","last_became_peered":"2023-07-06T02:22:53.424498+0000","last_unstale":"2023-07-06T02:22:58.623746+0000","last_undegraded":"2023-07-06T02:22:58.623746+0000","last_fullsized":"2023-07-06T02:22:58.623746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:55:40.483266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462649+0000","last_change":"2023-07-06T02:22:53.424739+0000","last_active":"2023-07-06T02:22:58.462649+0000","last_peered":"2023-07-06T02:22:58.462649+0000","last_clean":"2023-07-06T02:22:58.462649+0000","last_became_active":"2023-07-06T02:22:53.424382+0000","last_became_peered":"2023-07-06T02:22:53.424382+0000","last_unstale":"2023-07-06T02:22:58.462649+0000","last_undegraded":"2023-07-06T02:22:58.462649+0000","last_fullsized":"2023-07-06T02:22:58.462649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:08:50.637616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462913+0000","last_change":"2023-07-06T02:22:53.435357+0000","last_active":"2023-07-06T02:22:58.462913+0000","last_peered":"2023-07-06T02:22:58.462913+0000","last_clean":"2023-07-06T02:22:58.462913+0000","last_became_active":"2023-07-06T02:22:53.435068+0000","last_became_peered":"2023-07-06T02:22:53.435068+0000","last_unstale":"2023-07-06T02:22:58.462913+0000","last_undegraded":"2023-07-06T02:22:58.462913+0000","last_fullsized":"2023-07-06T02:22:58.462913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:05.764438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642840+0000","last_change":"2023-07-06T02:22:53.419307+0000","last_active":"2023-07-06T02:22:59.642840+0000","last_peered":"2023-07-06T02:22:59.642840+0000","last_clean":"2023-07-06T02:22:59.642840+0000","last_became_active":"2023-07-06T02:22:53.419010+0000","last_became_peered":"2023-07-06T02:22:53.419010+0000","last_unstale":"2023-07-06T02:22:59.642840+0000","last_undegraded":"2023-07-06T02:22:59.642840+0000","last_fullsized":"2023-07-06T02:22:59.642840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:52.740513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545431+0000","last_change":"2023-07-06T02:22:53.427932+0000","last_active":"2023-07-06T02:22:58.545431+0000","last_peered":"2023-07-06T02:22:58.545431+0000","last_clean":"2023-07-06T02:22:58.545431+0000","last_became_active":"2023-07-06T02:22:53.426280+0000","last_became_peered":"2023-07-06T02:22:53.426280+0000","last_unstale":"2023-07-06T02:22:58.545431+0000","last_undegraded":"2023-07-06T02:22:58.545431+0000","last_fullsized":"2023-07-06T02:22:58.545431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:05.507588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622966+0000","last_change":"2023-07-06T02:22:53.412012+0000","last_active":"2023-07-06T02:22:58.622966+0000","last_peered":"2023-07-06T02:22:58.622966+0000","last_clean":"2023-07-06T02:22:58.622966+0000","last_became_active":"2023-07-06T02:22:53.411675+0000","last_became_peered":"2023-07-06T02:22:53.411675+0000","last_unstale":"2023-07-06T02:22:58.622966+0000","last_undegraded":"2023-07-06T02:22:58.622966+0000","last_fullsized":"2023-07-06T02:22:58.622966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:13:54.565351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:53.432526+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:53.432240+0000","last_became_peered":"2023-07-06T02:22:53.432240+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623666+0000","last_change":"2023-07-06T02:22:53.419854+0000","last_active":"2023-07-06T02:22:58.623666+0000","last_peered":"2023-07-06T02:22:58.623666+0000","last_clean":"2023-07-06T02:22:58.623666+0000","last_became_active":"2023-07-06T02:22:53.419569+0000","last_became_peered":"2023-07-06T02:22:53.419569+0000","last_unstale":"2023-07-06T02:22:58.623666+0000","last_undegraded":"2023-07-06T02:22:58.623666+0000","last_fullsized":"2023-07-06T02:22:58.623666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:44:47.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463157+0000","last_change":"2023-07-06T02:22:53.422202+0000","last_active":"2023-07-06T02:22:58.463157+0000","last_peered":"2023-07-06T02:22:58.463157+0000","last_clean":"2023-07-06T02:22:58.463157+0000","last_became_active":"2023-07-06T02:22:53.421981+0000","last_became_peered":"2023-07-06T02:22:53.421981+0000","last_unstale":"2023-07-06T02:22:58.463157+0000","last_undegraded":"2023-07-06T02:22:58.463157+0000","last_fullsized":"2023-07-06T02:22:58.463157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:55.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623252+0000","last_change":"2023-07-06T02:22:53.421294+0000","last_active":"2023-07-06T02:22:58.623252+0000","last_peered":"2023-07-06T02:22:58.623252+0000","last_clean":"2023-07-06T02:22:58.623252+0000","last_became_active":"2023-07-06T02:22:53.420987+0000","last_became_peered":"2023-07-06T02:22:53.420987+0000","last_unstale":"2023-07-06T02:22:58.623252+0000","last_undegraded":"2023-07-06T02:22:58.623252+0000","last_fullsized":"2023-07-06T02:22:58.623252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:07:23.523735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545995+0000","last_change":"2023-07-06T02:22:53.423456+0000","last_active":"2023-07-06T02:22:58.545995+0000","last_peered":"2023-07-06T02:22:58.545995+0000","last_clean":"2023-07-06T02:22:58.545995+0000","last_became_active":"2023-07-06T02:22:53.423131+0000","last_became_peered":"2023-07-06T02:22:53.423131+0000","last_unstale":"2023-07-06T02:22:58.545995+0000","last_undegraded":"2023-07-06T02:22:58.545995+0000","last_fullsized":"2023-07-06T02:22:58.545995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641976+0000","last_change":"2023-07-06T02:22:53.424047+0000","last_active":"2023-07-06T02:22:59.641976+0000","last_peered":"2023-07-06T02:22:59.641976+0000","last_clean":"2023-07-06T02:22:59.641976+0000","last_became_active":"2023-07-06T02:22:53.423692+0000","last_became_peered":"2023-07-06T02:22:53.423692+0000","last_unstale":"2023-07-06T02:22:59.641976+0000","last_undegraded":"2023-07-06T02:22:59.641976+0000","last_fullsized":"2023-07-06T02:22:59.641976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.432531+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.432245+0000","last_became_peered":"2023-07-06T02:22:53.432245+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:09:56.578120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545462+0000","last_change":"2023-07-06T02:22:53.426016+0000","last_active":"2023-07-06T02:22:58.545462+0000","last_peered":"2023-07-06T02:22:58.545462+0000","last_clean":"2023-07-06T02:22:58.545462+0000","last_became_active":"2023-07-06T02:22:53.417030+0000","last_became_peered":"2023-07-06T02:22:53.417030+0000","last_unstale":"2023-07-06T02:22:58.545462+0000","last_undegraded":"2023-07-06T02:22:58.545462+0000","last_fullsized":"2023-07-06T02:22:58.545462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497209+0000","last_change":"2023-07-06T02:22:53.422907+0000","last_active":"2023-07-06T02:22:58.497209+0000","last_peered":"2023-07-06T02:22:58.497209+0000","last_clean":"2023-07-06T02:22:58.497209+0000","last_became_active":"2023-07-06T02:22:53.422617+0000","last_became_peered":"2023-07-06T02:22:53.422617+0000","last_unstale":"2023-07-06T02:22:58.497209+0000","last_undegraded":"2023-07-06T02:22:58.497209+0000","last_fullsized":"2023-07-06T02:22:58.497209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:41.517576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497094+0000","last_change":"2023-07-06T02:22:53.422413+0000","last_active":"2023-07-06T02:22:58.497094+0000","last_peered":"2023-07-06T02:22:58.497094+0000","last_clean":"2023-07-06T02:22:58.497094+0000","last_became_active":"2023-07-06T02:22:53.422025+0000","last_became_peered":"2023-07-06T02:22:53.422025+0000","last_unstale":"2023-07-06T02:22:58.497094+0000","last_undegraded":"2023-07-06T02:22:58.497094+0000","last_fullsized":"2023-07-06T02:22:58.497094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:23.425050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643229+0000","last_change":"2023-07-06T02:22:53.434688+0000","last_active":"2023-07-06T02:22:59.643229+0000","last_peered":"2023-07-06T02:22:59.643229+0000","last_clean":"2023-07-06T02:22:59.643229+0000","last_became_active":"2023-07-06T02:22:53.434423+0000","last_became_peered":"2023-07-06T02:22:53.434423+0000","last_unstale":"2023-07-06T02:22:59.643229+0000","last_undegraded":"2023-07-06T02:22:59.643229+0000","last_fullsized":"2023-07-06T02:22:59.643229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:11.004232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623583+0000","last_change":"2023-07-06T02:22:57.463283+0000","last_active":"2023-07-06T02:22:58.623583+0000","last_peered":"2023-07-06T02:22:58.623583+0000","last_clean":"2023-07-06T02:22:58.623583+0000","last_became_active":"2023-07-06T02:22:57.462913+0000","last_became_peered":"2023-07-06T02:22:57.462913+0000","last_unstale":"2023-07-06T02:22:58.623583+0000","last_undegraded":"2023-07-06T02:22:58.623583+0000","last_fullsized":"2023-07-06T02:22:58.623583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:29.980455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623061+0000","last_change":"2023-07-06T02:22:52.423689+0000","last_active":"2023-07-06T02:22:58.623061+0000","last_peered":"2023-07-06T02:22:58.623061+0000","last_clean":"2023-07-06T02:22:58.623061+0000","last_became_active":"2023-07-06T02:22:52.423464+0000","last_became_peered":"2023-07-06T02:22:52.423464+0000","last_unstale":"2023-07-06T02:22:58.623061+0000","last_undegraded":"2023-07-06T02:22:58.623061+0000","last_fullsized":"2023-07-06T02:22:58.623061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:34.698538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463653+0000","last_change":"2023-07-06T02:22:53.426231+0000","last_active":"2023-07-06T02:22:58.463653+0000","last_peered":"2023-07-06T02:22:58.463653+0000","last_clean":"2023-07-06T02:22:58.463653+0000","last_became_active":"2023-07-06T02:22:53.425923+0000","last_became_peered":"2023-07-06T02:22:53.425923+0000","last_unstale":"2023-07-06T02:22:58.463653+0000","last_undegraded":"2023-07-06T02:22:58.463653+0000","last_fullsized":"2023-07-06T02:22:58.463653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:22.845352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463216+0000","last_change":"2023-07-06T02:22:57.461661+0000","last_active":"2023-07-06T02:22:58.463216+0000","last_peered":"2023-07-06T02:22:58.463216+0000","last_clean":"2023-07-06T02:22:58.463216+0000","last_became_active":"2023-07-06T02:22:57.461262+0000","last_became_peered":"2023-07-06T02:22:57.461262+0000","last_unstale":"2023-07-06T02:22:58.463216+0000","last_undegraded":"2023-07-06T02:22:58.463216+0000","last_fullsized":"2023-07-06T02:22:58.463216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:05.846802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496610+0000","last_change":"2023-07-06T02:22:52.424261+0000","last_active":"2023-07-06T02:22:58.496610+0000","last_peered":"2023-07-06T02:22:58.496610+0000","last_clean":"2023-07-06T02:22:58.496610+0000","last_became_active":"2023-07-06T02:22:52.423984+0000","last_became_peered":"2023-07-06T02:22:52.423984+0000","last_unstale":"2023-07-06T02:22:58.496610+0000","last_undegraded":"2023-07-06T02:22:58.496610+0000","last_fullsized":"2023-07-06T02:22:58.496610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:56.520485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642313+0000","last_change":"2023-07-06T02:22:53.417163+0000","last_active":"2023-07-06T02:22:59.642313+0000","last_peered":"2023-07-06T02:22:59.642313+0000","last_clean":"2023-07-06T02:22:59.642313+0000","last_became_active":"2023-07-06T02:22:53.416875+0000","last_became_peered":"2023-07-06T02:22:53.416875+0000","last_unstale":"2023-07-06T02:22:59.642313+0000","last_undegraded":"2023-07-06T02:22:59.642313+0000","last_fullsized":"2023-07-06T02:22:59.642313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642441+0000","last_change":"2023-07-06T02:22:57.455750+0000","last_active":"2023-07-06T02:22:59.642441+0000","last_peered":"2023-07-06T02:22:59.642441+0000","last_clean":"2023-07-06T02:22:59.642441+0000","last_became_active":"2023-07-06T02:22:57.455331+0000","last_became_peered":"2023-07-06T02:22:57.455331+0000","last_unstale":"2023-07-06T02:22:59.642441+0000","last_undegraded":"2023-07-06T02:22:59.642441+0000","last_fullsized":"2023-07-06T02:22:59.642441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462903+0000","last_change":"2023-07-06T02:22:52.416577+0000","last_active":"2023-07-06T02:22:58.462903+0000","last_peered":"2023-07-06T02:22:58.462903+0000","last_clean":"2023-07-06T02:22:58.462903+0000","last_became_active":"2023-07-06T02:22:52.416144+0000","last_became_peered":"2023-07-06T02:22:52.416144+0000","last_unstale":"2023-07-06T02:22:58.462903+0000","last_undegraded":"2023-07-06T02:22:58.462903+0000","last_fullsized":"2023-07-06T02:22:58.462903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:03:22.897984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463486+0000","last_change":"2023-07-06T02:22:53.429089+0000","last_active":"2023-07-06T02:22:58.463486+0000","last_peered":"2023-07-06T02:22:58.463486+0000","last_clean":"2023-07-06T02:22:58.463486+0000","last_became_active":"2023-07-06T02:22:53.428809+0000","last_became_peered":"2023-07-06T02:22:53.428809+0000","last_unstale":"2023-07-06T02:22:58.463486+0000","last_undegraded":"2023-07-06T02:22:58.463486+0000","last_fullsized":"2023-07-06T02:22:58.463486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:30:16.217797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462711+0000","last_change":"2023-07-06T02:22:57.455092+0000","last_active":"2023-07-06T02:22:58.462711+0000","last_peered":"2023-07-06T02:22:58.462711+0000","last_clean":"2023-07-06T02:22:58.462711+0000","last_became_active":"2023-07-06T02:22:57.454707+0000","last_became_peered":"2023-07-06T02:22:57.454707+0000","last_unstale":"2023-07-06T02:22:58.462711+0000","last_undegraded":"2023-07-06T02:22:58.462711+0000","last_fullsized":"2023-07-06T02:22:58.462711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:09:16.226734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:52.428104+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:52.427863+0000","last_became_peered":"2023-07-06T02:22:52.427863+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:52.064862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.424797+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.424474+0000","last_became_peered":"2023-07-06T02:22:53.424474+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:31:13.843201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545356+0000","last_change":"2023-07-06T02:22:57.457309+0000","last_active":"2023-07-06T02:22:58.545356+0000","last_peered":"2023-07-06T02:22:58.545356+0000","last_clean":"2023-07-06T02:22:58.545356+0000","last_became_active":"2023-07-06T02:22:57.457008+0000","last_became_peered":"2023-07-06T02:22:57.457008+0000","last_unstale":"2023-07-06T02:22:58.545356+0000","last_undegraded":"2023-07-06T02:22:58.545356+0000","last_fullsized":"2023-07-06T02:22:58.545356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:04.437413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643326+0000","last_change":"2023-07-06T02:22:52.423048+0000","last_active":"2023-07-06T02:22:59.643326+0000","last_peered":"2023-07-06T02:22:59.643326+0000","last_clean":"2023-07-06T02:22:59.643326+0000","last_became_active":"2023-07-06T02:22:52.422801+0000","last_became_peered":"2023-07-06T02:22:52.422801+0000","last_unstale":"2023-07-06T02:22:59.643326+0000","last_undegraded":"2023-07-06T02:22:59.643326+0000","last_fullsized":"2023-07-06T02:22:59.643326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:29.807668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462478+0000","last_change":"2023-07-06T02:22:53.434004+0000","last_active":"2023-07-06T02:22:58.462478+0000","last_peered":"2023-07-06T02:22:58.462478+0000","last_clean":"2023-07-06T02:22:58.462478+0000","last_became_active":"2023-07-06T02:22:53.433739+0000","last_became_peered":"2023-07-06T02:22:53.433739+0000","last_unstale":"2023-07-06T02:22:58.462478+0000","last_undegraded":"2023-07-06T02:22:58.462478+0000","last_fullsized":"2023-07-06T02:22:58.462478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463415+0000","last_change":"2023-07-06T02:22:57.452160+0000","last_active":"2023-07-06T02:22:58.463415+0000","last_peered":"2023-07-06T02:22:58.463415+0000","last_clean":"2023-07-06T02:22:58.463415+0000","last_became_active":"2023-07-06T02:22:57.451507+0000","last_became_peered":"2023-07-06T02:22:57.451507+0000","last_unstale":"2023-07-06T02:22:58.463415+0000","last_undegraded":"2023-07-06T02:22:58.463415+0000","last_fullsized":"2023-07-06T02:22:58.463415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:10:18.060753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546029+0000","last_change":"2023-07-06T02:22:52.419096+0000","last_active":"2023-07-06T02:22:58.546029+0000","last_peered":"2023-07-06T02:22:58.546029+0000","last_clean":"2023-07-06T02:22:58.546029+0000","last_became_active":"2023-07-06T02:22:52.418844+0000","last_became_peered":"2023-07-06T02:22:52.418844+0000","last_unstale":"2023-07-06T02:22:58.546029+0000","last_undegraded":"2023-07-06T02:22:58.546029+0000","last_fullsized":"2023-07-06T02:22:58.546029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:14.974840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496797+0000","last_change":"2023-07-06T02:22:53.425547+0000","last_active":"2023-07-06T02:22:58.496797+0000","last_peered":"2023-07-06T02:22:58.496797+0000","last_clean":"2023-07-06T02:22:58.496797+0000","last_became_active":"2023-07-06T02:22:53.425239+0000","last_became_peered":"2023-07-06T02:22:53.425239+0000","last_unstale":"2023-07-06T02:22:58.496797+0000","last_undegraded":"2023-07-06T02:22:58.496797+0000","last_fullsized":"2023-07-06T02:22:58.496797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:32.244431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496944+0000","last_change":"2023-07-06T02:22:57.453836+0000","last_active":"2023-07-06T02:22:58.496944+0000","last_peered":"2023-07-06T02:22:58.496944+0000","last_clean":"2023-07-06T02:22:58.496944+0000","last_became_active":"2023-07-06T02:22:57.451637+0000","last_became_peered":"2023-07-06T02:22:57.451637+0000","last_unstale":"2023-07-06T02:22:58.496944+0000","last_undegraded":"2023-07-06T02:22:58.496944+0000","last_fullsized":"2023-07-06T02:22:58.496944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:45.805424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:23:03.321857+0000","last_change":"2023-07-06T02:22:52.421021+0000","last_active":"2023-07-06T02:23:03.321857+0000","last_peered":"2023-07-06T02:23:03.321857+0000","last_clean":"2023-07-06T02:23:03.321857+0000","last_became_active":"2023-07-06T02:22:52.420768+0000","last_became_peered":"2023-07-06T02:22:52.420768+0000","last_unstale":"2023-07-06T02:23:03.321857+0000","last_undegraded":"2023-07-06T02:23:03.321857+0000","last_fullsized":"2023-07-06T02:23:03.321857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:10:06.879321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623414+0000","last_change":"2023-07-06T02:22:53.420729+0000","last_active":"2023-07-06T02:22:58.623414+0000","last_peered":"2023-07-06T02:22:58.623414+0000","last_clean":"2023-07-06T02:22:58.623414+0000","last_became_active":"2023-07-06T02:22:53.418685+0000","last_became_peered":"2023-07-06T02:22:53.418685+0000","last_unstale":"2023-07-06T02:22:58.623414+0000","last_undegraded":"2023-07-06T02:22:58.623414+0000","last_fullsized":"2023-07-06T02:22:58.623414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:47.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643458+0000","last_change":"2023-07-06T02:22:57.455310+0000","last_active":"2023-07-06T02:22:59.643458+0000","last_peered":"2023-07-06T02:22:59.643458+0000","last_clean":"2023-07-06T02:22:59.643458+0000","last_became_active":"2023-07-06T02:22:57.454518+0000","last_became_peered":"2023-07-06T02:22:57.454518+0000","last_unstale":"2023-07-06T02:22:59.643458+0000","last_undegraded":"2023-07-06T02:22:59.643458+0000","last_fullsized":"2023-07-06T02:22:59.643458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:57.100343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545849+0000","last_change":"2023-07-06T02:22:52.421399+0000","last_active":"2023-07-06T02:22:58.545849+0000","last_peered":"2023-07-06T02:22:58.545849+0000","last_clean":"2023-07-06T02:22:58.545849+0000","last_became_active":"2023-07-06T02:22:52.420062+0000","last_became_peered":"2023-07-06T02:22:52.420062+0000","last_unstale":"2023-07-06T02:22:58.545849+0000","last_undegraded":"2023-07-06T02:22:58.545849+0000","last_fullsized":"2023-07-06T02:22:58.545849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:53.425567+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:53.425272+0000","last_became_peered":"2023-07-06T02:22:53.425272+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:45.959463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463853+0000","last_change":"2023-07-06T02:22:57.452727+0000","last_active":"2023-07-06T02:22:58.463853+0000","last_peered":"2023-07-06T02:22:58.463853+0000","last_clean":"2023-07-06T02:22:58.463853+0000","last_became_active":"2023-07-06T02:22:57.452043+0000","last_became_peered":"2023-07-06T02:22:57.452043+0000","last_unstale":"2023-07-06T02:22:58.463853+0000","last_undegraded":"2023-07-06T02:22:58.463853+0000","last_fullsized":"2023-07-06T02:22:58.463853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:26.074085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642190+0000","last_change":"2023-07-06T02:22:52.420802+0000","last_active":"2023-07-06T02:22:59.642190+0000","last_peered":"2023-07-06T02:22:59.642190+0000","last_clean":"2023-07-06T02:22:59.642190+0000","last_became_active":"2023-07-06T02:22:52.420494+0000","last_became_peered":"2023-07-06T02:22:52.420494+0000","last_unstale":"2023-07-06T02:22:59.642190+0000","last_undegraded":"2023-07-06T02:22:59.642190+0000","last_fullsized":"2023-07-06T02:22:59.642190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:04.500268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643228+0000","last_change":"2023-07-06T02:22:53.432233+0000","last_active":"2023-07-06T02:22:59.643228+0000","last_peered":"2023-07-06T02:22:59.643228+0000","last_clean":"2023-07-06T02:22:59.643228+0000","last_became_active":"2023-07-06T02:22:53.431893+0000","last_became_peered":"2023-07-06T02:22:53.431893+0000","last_unstale":"2023-07-06T02:22:59.643228+0000","last_undegraded":"2023-07-06T02:22:59.643228+0000","last_fullsized":"2023-07-06T02:22:59.643228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463256+0000","last_change":"2023-07-06T02:22:57.455213+0000","last_active":"2023-07-06T02:22:58.463256+0000","last_peered":"2023-07-06T02:22:58.463256+0000","last_clean":"2023-07-06T02:22:58.463256+0000","last_became_active":"2023-07-06T02:22:57.454800+0000","last_became_peered":"2023-07-06T02:22:57.454800+0000","last_unstale":"2023-07-06T02:22:58.463256+0000","last_undegraded":"2023-07-06T02:22:58.463256+0000","last_fullsized":"2023-07-06T02:22:58.463256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:00.449720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545483+0000","last_change":"2023-07-06T02:22:52.421475+0000","last_active":"2023-07-06T02:22:58.545483+0000","last_peered":"2023-07-06T02:22:58.545483+0000","last_clean":"2023-07-06T02:22:58.545483+0000","last_became_active":"2023-07-06T02:22:52.421267+0000","last_became_peered":"2023-07-06T02:22:52.421267+0000","last_unstale":"2023-07-06T02:22:58.545483+0000","last_undegraded":"2023-07-06T02:22:58.545483+0000","last_fullsized":"2023-07-06T02:22:58.545483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:09.996912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642587+0000","last_change":"2023-07-06T02:22:53.420060+0000","last_active":"2023-07-06T02:22:59.642587+0000","last_peered":"2023-07-06T02:22:59.642587+0000","last_clean":"2023-07-06T02:22:59.642587+0000","last_became_active":"2023-07-06T02:22:53.419347+0000","last_became_peered":"2023-07-06T02:22:53.419347+0000","last_unstale":"2023-07-06T02:22:59.642587+0000","last_undegraded":"2023-07-06T02:22:59.642587+0000","last_fullsized":"2023-07-06T02:22:59.642587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643160+0000","last_change":"2023-07-06T02:22:57.453231+0000","last_active":"2023-07-06T02:22:59.643160+0000","last_peered":"2023-07-06T02:22:59.643160+0000","last_clean":"2023-07-06T02:22:59.643160+0000","last_became_active":"2023-07-06T02:22:57.452776+0000","last_became_peered":"2023-07-06T02:22:57.452776+0000","last_unstale":"2023-07-06T02:22:59.643160+0000","last_undegraded":"2023-07-06T02:22:59.643160+0000","last_fullsized":"2023-07-06T02:22:59.643160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:09.535749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462794+0000","last_change":"2023-07-06T02:22:52.417281+0000","last_active":"2023-07-06T02:22:58.462794+0000","last_peered":"2023-07-06T02:22:58.462794+0000","last_clean":"2023-07-06T02:22:58.462794+0000","last_became_active":"2023-07-06T02:22:52.417011+0000","last_became_peered":"2023-07-06T02:22:52.417011+0000","last_unstale":"2023-07-06T02:22:58.462794+0000","last_undegraded":"2023-07-06T02:22:58.462794+0000","last_fullsized":"2023-07-06T02:22:58.462794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:56.885245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463676+0000","last_change":"2023-07-06T02:22:53.433973+0000","last_active":"2023-07-06T02:22:58.463676+0000","last_peered":"2023-07-06T02:22:58.463676+0000","last_clean":"2023-07-06T02:22:58.463676+0000","last_became_active":"2023-07-06T02:22:53.433561+0000","last_became_peered":"2023-07-06T02:22:53.433561+0000","last_unstale":"2023-07-06T02:22:58.463676+0000","last_undegraded":"2023-07-06T02:22:58.463676+0000","last_fullsized":"2023-07-06T02:22:58.463676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:36.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545710+0000","last_change":"2023-07-06T02:22:57.455016+0000","last_active":"2023-07-06T02:22:58.545710+0000","last_peered":"2023-07-06T02:22:58.545710+0000","last_clean":"2023-07-06T02:22:58.545710+0000","last_became_active":"2023-07-06T02:22:57.454627+0000","last_became_peered":"2023-07-06T02:22:57.454627+0000","last_unstale":"2023-07-06T02:22:58.545710+0000","last_undegraded":"2023-07-06T02:22:58.545710+0000","last_fullsized":"2023-07-06T02:22:58.545710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546232+0000","last_change":"2023-07-06T02:22:52.420410+0000","last_active":"2023-07-06T02:22:58.546232+0000","last_peered":"2023-07-06T02:22:58.546232+0000","last_clean":"2023-07-06T02:22:58.546232+0000","last_became_active":"2023-07-06T02:22:52.420089+0000","last_became_peered":"2023-07-06T02:22:52.420089+0000","last_unstale":"2023-07-06T02:22:58.546232+0000","last_undegraded":"2023-07-06T02:22:58.546232+0000","last_fullsized":"2023-07-06T02:22:58.546232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642490+0000","last_change":"2023-07-06T02:22:53.432339+0000","last_active":"2023-07-06T02:22:59.642490+0000","last_peered":"2023-07-06T02:22:59.642490+0000","last_clean":"2023-07-06T02:22:59.642490+0000","last_became_active":"2023-07-06T02:22:53.431993+0000","last_became_peered":"2023-07-06T02:22:53.431993+0000","last_unstale":"2023-07-06T02:22:59.642490+0000","last_undegraded":"2023-07-06T02:22:59.642490+0000","last_fullsized":"2023-07-06T02:22:59.642490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462370+0000","last_change":"2023-07-06T02:22:57.457792+0000","last_active":"2023-07-06T02:22:58.462370+0000","last_peered":"2023-07-06T02:22:58.462370+0000","last_clean":"2023-07-06T02:22:58.462370+0000","last_became_active":"2023-07-06T02:22:57.457396+0000","last_became_peered":"2023-07-06T02:22:57.457396+0000","last_unstale":"2023-07-06T02:22:58.462370+0000","last_undegraded":"2023-07-06T02:22:58.462370+0000","last_fullsized":"2023-07-06T02:22:58.462370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:17.206072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546155+0000","last_change":"2023-07-06T02:22:52.421892+0000","last_active":"2023-07-06T02:22:58.546155+0000","last_peered":"2023-07-06T02:22:58.546155+0000","last_clean":"2023-07-06T02:22:58.546155+0000","last_became_active":"2023-07-06T02:22:52.421613+0000","last_became_peered":"2023-07-06T02:22:52.421613+0000","last_unstale":"2023-07-06T02:22:58.546155+0000","last_undegraded":"2023-07-06T02:22:58.546155+0000","last_fullsized":"2023-07-06T02:22:58.546155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622924+0000","last_change":"2023-07-06T02:22:53.422219+0000","last_active":"2023-07-06T02:22:58.622924+0000","last_peered":"2023-07-06T02:22:58.622924+0000","last_clean":"2023-07-06T02:22:58.622924+0000","last_became_active":"2023-07-06T02:22:53.421934+0000","last_became_peered":"2023-07-06T02:22:53.421934+0000","last_unstale":"2023-07-06T02:22:58.622924+0000","last_undegraded":"2023-07-06T02:22:58.622924+0000","last_fullsized":"2023-07-06T02:22:58.622924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:14:41.243585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462824+0000","last_change":"2023-07-06T02:22:57.450611+0000","last_active":"2023-07-06T02:22:58.462824+0000","last_peered":"2023-07-06T02:22:58.462824+0000","last_clean":"2023-07-06T02:22:58.462824+0000","last_became_active":"2023-07-06T02:22:57.450255+0000","last_became_peered":"2023-07-06T02:22:57.450255+0000","last_unstale":"2023-07-06T02:22:58.462824+0000","last_undegraded":"2023-07-06T02:22:58.462824+0000","last_fullsized":"2023-07-06T02:22:58.462824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:04.461198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545892+0000","last_change":"2023-07-06T02:22:52.420450+0000","last_active":"2023-07-06T02:22:58.545892+0000","last_peered":"2023-07-06T02:22:58.545892+0000","last_clean":"2023-07-06T02:22:58.545892+0000","last_became_active":"2023-07-06T02:22:52.420171+0000","last_became_peered":"2023-07-06T02:22:52.420171+0000","last_unstale":"2023-07-06T02:22:58.545892+0000","last_undegraded":"2023-07-06T02:22:58.545892+0000","last_fullsized":"2023-07-06T02:22:58.545892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:35.849912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462735+0000","last_change":"2023-07-06T02:22:53.425285+0000","last_active":"2023-07-06T02:22:58.462735+0000","last_peered":"2023-07-06T02:22:58.462735+0000","last_clean":"2023-07-06T02:22:58.462735+0000","last_became_active":"2023-07-06T02:22:53.424994+0000","last_became_peered":"2023-07-06T02:22:53.424994+0000","last_unstale":"2023-07-06T02:22:58.462735+0000","last_undegraded":"2023-07-06T02:22:58.462735+0000","last_fullsized":"2023-07-06T02:22:58.462735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.580209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462875+0000","last_change":"2023-07-06T02:22:57.459471+0000","last_active":"2023-07-06T02:22:58.462875+0000","last_peered":"2023-07-06T02:22:58.462875+0000","last_clean":"2023-07-06T02:22:58.462875+0000","last_became_active":"2023-07-06T02:22:57.459086+0000","last_became_peered":"2023-07-06T02:22:57.459086+0000","last_unstale":"2023-07-06T02:22:58.462875+0000","last_undegraded":"2023-07-06T02:22:58.462875+0000","last_fullsized":"2023-07-06T02:22:58.462875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:32.995367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643074+0000","last_change":"2023-07-06T02:22:52.419379+0000","last_active":"2023-07-06T02:22:59.643074+0000","last_peered":"2023-07-06T02:22:59.643074+0000","last_clean":"2023-07-06T02:22:59.643074+0000","last_became_active":"2023-07-06T02:22:52.419100+0000","last_became_peered":"2023-07-06T02:22:52.419100+0000","last_unstale":"2023-07-06T02:22:59.643074+0000","last_undegraded":"2023-07-06T02:22:59.643074+0000","last_fullsized":"2023-07-06T02:22:59.643074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:15.612393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642483+0000","last_change":"2023-07-06T02:22:53.423846+0000","last_active":"2023-07-06T02:22:59.642483+0000","last_peered":"2023-07-06T02:22:59.642483+0000","last_clean":"2023-07-06T02:22:59.642483+0000","last_became_active":"2023-07-06T02:22:53.423585+0000","last_became_peered":"2023-07-06T02:22:53.423585+0000","last_unstale":"2023-07-06T02:22:59.642483+0000","last_undegraded":"2023-07-06T02:22:59.642483+0000","last_fullsized":"2023-07-06T02:22:59.642483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:05.040887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462786+0000","last_change":"2023-07-06T02:22:57.450230+0000","last_active":"2023-07-06T02:22:58.462786+0000","last_peered":"2023-07-06T02:22:58.462786+0000","last_clean":"2023-07-06T02:22:58.462786+0000","last_became_active":"2023-07-06T02:22:57.449841+0000","last_became_peered":"2023-07-06T02:22:57.449841+0000","last_unstale":"2023-07-06T02:22:58.462786+0000","last_undegraded":"2023-07-06T02:22:58.462786+0000","last_fullsized":"2023-07-06T02:22:58.462786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:26:07.959918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462612+0000","last_change":"2023-07-06T02:22:52.418001+0000","last_active":"2023-07-06T02:22:58.462612+0000","last_peered":"2023-07-06T02:22:58.462612+0000","last_clean":"2023-07-06T02:22:58.462612+0000","last_became_active":"2023-07-06T02:22:52.417737+0000","last_became_peered":"2023-07-06T02:22:52.417737+0000","last_unstale":"2023-07-06T02:22:58.462612+0000","last_undegraded":"2023-07-06T02:22:58.462612+0000","last_fullsized":"2023-07-06T02:22:58.462612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623068+0000","last_change":"2023-07-06T02:22:53.418072+0000","last_active":"2023-07-06T02:22:58.623068+0000","last_peered":"2023-07-06T02:22:58.623068+0000","last_clean":"2023-07-06T02:22:58.623068+0000","last_became_active":"2023-07-06T02:22:53.416070+0000","last_became_peered":"2023-07-06T02:22:53.416070+0000","last_unstale":"2023-07-06T02:22:58.623068+0000","last_undegraded":"2023-07-06T02:22:58.623068+0000","last_fullsized":"2023-07-06T02:22:58.623068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:39:55.241653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497345+0000","last_change":"2023-07-06T02:22:57.451350+0000","last_active":"2023-07-06T02:22:58.497345+0000","last_peered":"2023-07-06T02:22:58.497345+0000","last_clean":"2023-07-06T02:22:58.497345+0000","last_became_active":"2023-07-06T02:22:57.450988+0000","last_became_peered":"2023-07-06T02:22:57.450988+0000","last_unstale":"2023-07-06T02:22:58.497345+0000","last_undegraded":"2023-07-06T02:22:58.497345+0000","last_fullsized":"2023-07-06T02:22:58.497345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:08:02.280780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462907+0000","last_change":"2023-07-06T02:22:52.423191+0000","last_active":"2023-07-06T02:22:58.462907+0000","last_peered":"2023-07-06T02:22:58.462907+0000","last_clean":"2023-07-06T02:22:58.462907+0000","last_became_active":"2023-07-06T02:22:52.422863+0000","last_became_peered":"2023-07-06T02:22:52.422863+0000","last_unstale":"2023-07-06T02:22:58.462907+0000","last_undegraded":"2023-07-06T02:22:58.462907+0000","last_fullsized":"2023-07-06T02:22:58.462907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:22.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623436+0000","last_change":"2023-07-06T02:22:53.419350+0000","last_active":"2023-07-06T02:22:58.623436+0000","last_peered":"2023-07-06T02:22:58.623436+0000","last_clean":"2023-07-06T02:22:58.623436+0000","last_became_active":"2023-07-06T02:22:53.419024+0000","last_became_peered":"2023-07-06T02:22:53.419024+0000","last_unstale":"2023-07-06T02:22:58.623436+0000","last_undegraded":"2023-07-06T02:22:58.623436+0000","last_fullsized":"2023-07-06T02:22:58.623436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:33.746001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643344+0000","last_change":"2023-07-06T02:22:57.455212+0000","last_active":"2023-07-06T02:22:59.643344+0000","last_peered":"2023-07-06T02:22:59.643344+0000","last_clean":"2023-07-06T02:22:59.643344+0000","last_became_active":"2023-07-06T02:22:57.454773+0000","last_became_peered":"2023-07-06T02:22:57.454773+0000","last_unstale":"2023-07-06T02:22:59.643344+0000","last_undegraded":"2023-07-06T02:22:59.643344+0000","last_fullsized":"2023-07-06T02:22:59.643344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622698+0000","last_change":"2023-07-06T02:22:52.417326+0000","last_active":"2023-07-06T02:22:58.622698+0000","last_peered":"2023-07-06T02:22:58.622698+0000","last_clean":"2023-07-06T02:22:58.622698+0000","last_became_active":"2023-07-06T02:22:52.417065+0000","last_became_peered":"2023-07-06T02:22:52.417065+0000","last_unstale":"2023-07-06T02:22:58.622698+0000","last_undegraded":"2023-07-06T02:22:58.622698+0000","last_fullsized":"2023-07-06T02:22:58.622698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:35.235033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623747+0000","last_change":"2023-07-06T02:22:53.422511+0000","last_active":"2023-07-06T02:22:58.623747+0000","last_peered":"2023-07-06T02:22:58.623747+0000","last_clean":"2023-07-06T02:22:58.623747+0000","last_became_active":"2023-07-06T02:22:53.420266+0000","last_became_peered":"2023-07-06T02:22:53.420266+0000","last_unstale":"2023-07-06T02:22:58.623747+0000","last_undegraded":"2023-07-06T02:22:58.623747+0000","last_fullsized":"2023-07-06T02:22:58.623747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:05.329078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623647+0000","last_change":"2023-07-06T02:22:57.451156+0000","last_active":"2023-07-06T02:22:58.623647+0000","last_peered":"2023-07-06T02:22:58.623647+0000","last_clean":"2023-07-06T02:22:58.623647+0000","last_became_active":"2023-07-06T02:22:57.450784+0000","last_became_peered":"2023-07-06T02:22:57.450784+0000","last_unstale":"2023-07-06T02:22:58.623647+0000","last_undegraded":"2023-07-06T02:22:58.623647+0000","last_fullsized":"2023-07-06T02:22:58.623647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:15:13.002024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462668+0000","last_change":"2023-07-06T02:22:52.418727+0000","last_active":"2023-07-06T02:22:58.462668+0000","last_peered":"2023-07-06T02:22:58.462668+0000","last_clean":"2023-07-06T02:22:58.462668+0000","last_became_active":"2023-07-06T02:22:52.418454+0000","last_became_peered":"2023-07-06T02:22:52.418454+0000","last_unstale":"2023-07-06T02:22:58.462668+0000","last_undegraded":"2023-07-06T02:22:58.462668+0000","last_fullsized":"2023-07-06T02:22:58.462668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:05.425463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497348+0000","last_change":"2023-07-06T02:22:53.432617+0000","last_active":"2023-07-06T02:22:58.497348+0000","last_peered":"2023-07-06T02:22:58.497348+0000","last_clean":"2023-07-06T02:22:58.497348+0000","last_became_active":"2023-07-06T02:22:53.432332+0000","last_became_peered":"2023-07-06T02:22:53.432332+0000","last_unstale":"2023-07-06T02:22:58.497348+0000","last_undegraded":"2023-07-06T02:22:58.497348+0000","last_fullsized":"2023-07-06T02:22:58.497348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623091+0000","last_change":"2023-07-06T02:22:57.463354+0000","last_active":"2023-07-06T02:22:58.623091+0000","last_peered":"2023-07-06T02:22:58.623091+0000","last_clean":"2023-07-06T02:22:58.623091+0000","last_became_active":"2023-07-06T02:22:57.462934+0000","last_became_peered":"2023-07-06T02:22:57.462934+0000","last_unstale":"2023-07-06T02:22:58.623091+0000","last_undegraded":"2023-07-06T02:22:58.623091+0000","last_fullsized":"2023-07-06T02:22:58.623091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:23.690834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463354+0000","last_change":"2023-07-06T02:22:52.421219+0000","last_active":"2023-07-06T02:22:58.463354+0000","last_peered":"2023-07-06T02:22:58.463354+0000","last_clean":"2023-07-06T02:22:58.463354+0000","last_became_active":"2023-07-06T02:22:52.420927+0000","last_became_peered":"2023-07-06T02:22:52.420927+0000","last_unstale":"2023-07-06T02:22:58.463354+0000","last_undegraded":"2023-07-06T02:22:58.463354+0000","last_fullsized":"2023-07-06T02:22:58.463354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:17:11.011220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623582+0000","last_change":"2023-07-06T02:22:47.973696+0000","last_active":"2023-07-06T02:22:58.623582+0000","last_peered":"2023-07-06T02:22:58.623582+0000","last_clean":"2023-07-06T02:22:58.623582+0000","last_became_active":"2023-07-06T02:22:47.973358+0000","last_became_peered":"2023-07-06T02:22:47.973358+0000","last_unstale":"2023-07-06T02:22:58.623582+0000","last_undegraded":"2023-07-06T02:22:58.623582+0000","last_fullsized":"2023-07-06T02:22:58.623582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:26:55.060583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462551+0000","last_change":"2023-07-06T02:22:53.433182+0000","last_active":"2023-07-06T02:22:58.462551+0000","last_peered":"2023-07-06T02:22:58.462551+0000","last_clean":"2023-07-06T02:22:58.462551+0000","last_became_active":"2023-07-06T02:22:53.432900+0000","last_became_peered":"2023-07-06T02:22:53.432900+0000","last_unstale":"2023-07-06T02:22:58.462551+0000","last_undegraded":"2023-07-06T02:22:58.462551+0000","last_fullsized":"2023-07-06T02:22:58.462551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462417+0000","last_change":"2023-07-06T02:22:57.456985+0000","last_active":"2023-07-06T02:22:58.462417+0000","last_peered":"2023-07-06T02:22:58.462417+0000","last_clean":"2023-07-06T02:22:58.462417+0000","last_became_active":"2023-07-06T02:22:57.456594+0000","last_became_peered":"2023-07-06T02:22:57.456594+0000","last_unstale":"2023-07-06T02:22:58.462417+0000","last_undegraded":"2023-07-06T02:22:58.462417+0000","last_fullsized":"2023-07-06T02:22:58.462417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:44.241938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463270+0000","last_change":"2023-07-06T02:22:47.323237+0000","last_active":"2023-07-06T02:22:58.463270+0000","last_peered":"2023-07-06T02:22:58.463270+0000","last_clean":"2023-07-06T02:22:58.463270+0000","last_became_active":"2023-07-06T02:22:47.322763+0000","last_became_peered":"2023-07-06T02:22:47.322763+0000","last_unstale":"2023-07-06T02:22:58.463270+0000","last_undegraded":"2023-07-06T02:22:58.463270+0000","last_fullsized":"2023-07-06T02:22:58.463270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:51:10.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642236+0000","last_change":"2023-07-06T02:22:52.417097+0000","last_active":"2023-07-06T02:22:59.642236+0000","last_peered":"2023-07-06T02:22:59.642236+0000","last_clean":"2023-07-06T02:22:59.642236+0000","last_became_active":"2023-07-06T02:22:52.416824+0000","last_became_peered":"2023-07-06T02:22:52.416824+0000","last_unstale":"2023-07-06T02:22:59.642236+0000","last_undegraded":"2023-07-06T02:22:59.642236+0000","last_fullsized":"2023-07-06T02:22:59.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:35.415046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622712+0000","last_change":"2023-07-06T02:22:53.419559+0000","last_active":"2023-07-06T02:22:58.622712+0000","last_peered":"2023-07-06T02:22:58.622712+0000","last_clean":"2023-07-06T02:22:58.622712+0000","last_became_active":"2023-07-06T02:22:53.419069+0000","last_became_peered":"2023-07-06T02:22:53.419069+0000","last_unstale":"2023-07-06T02:22:58.622712+0000","last_undegraded":"2023-07-06T02:22:58.622712+0000","last_fullsized":"2023-07-06T02:22:58.622712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:22.045924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643230+0000","last_change":"2023-07-06T02:22:57.459369+0000","last_active":"2023-07-06T02:22:59.643230+0000","last_peered":"2023-07-06T02:22:59.643230+0000","last_clean":"2023-07-06T02:22:59.643230+0000","last_became_active":"2023-07-06T02:22:57.458965+0000","last_became_peered":"2023-07-06T02:22:57.458965+0000","last_unstale":"2023-07-06T02:22:59.643230+0000","last_undegraded":"2023-07-06T02:22:59.643230+0000","last_fullsized":"2023-07-06T02:22:59.643230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:40.450527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496831+0000","last_change":"2023-07-06T02:22:47.325598+0000","last_active":"2023-07-06T02:22:58.496831+0000","last_peered":"2023-07-06T02:22:58.496831+0000","last_clean":"2023-07-06T02:22:58.496831+0000","last_became_active":"2023-07-06T02:22:47.325315+0000","last_became_peered":"2023-07-06T02:22:47.325315+0000","last_unstale":"2023-07-06T02:22:58.496831+0000","last_undegraded":"2023-07-06T02:22:58.496831+0000","last_fullsized":"2023-07-06T02:22:58.496831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:36.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545612+0000","last_change":"2023-07-06T02:22:47.326882+0000","last_active":"2023-07-06T02:22:58.545612+0000","last_peered":"2023-07-06T02:22:58.545612+0000","last_clean":"2023-07-06T02:22:58.545612+0000","last_became_active":"2023-07-06T02:22:47.326511+0000","last_became_peered":"2023-07-06T02:22:47.326511+0000","last_unstale":"2023-07-06T02:22:58.545612+0000","last_undegraded":"2023-07-06T02:22:58.545612+0000","last_fullsized":"2023-07-06T02:22:58.545612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:46.985907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641990+0000","last_change":"2023-07-06T02:22:52.416206+0000","last_active":"2023-07-06T02:22:59.641990+0000","last_peered":"2023-07-06T02:22:59.641990+0000","last_clean":"2023-07-06T02:22:59.641990+0000","last_became_active":"2023-07-06T02:22:52.415905+0000","last_became_peered":"2023-07-06T02:22:52.415905+0000","last_unstale":"2023-07-06T02:22:59.641990+0000","last_undegraded":"2023-07-06T02:22:59.641990+0000","last_fullsized":"2023-07-06T02:22:59.641990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:59.401932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642535+0000","last_change":"2023-07-06T02:22:53.423988+0000","last_active":"2023-07-06T02:22:59.642535+0000","last_peered":"2023-07-06T02:22:59.642535+0000","last_clean":"2023-07-06T02:22:59.642535+0000","last_became_active":"2023-07-06T02:22:53.423711+0000","last_became_peered":"2023-07-06T02:22:53.423711+0000","last_unstale":"2023-07-06T02:22:59.642535+0000","last_undegraded":"2023-07-06T02:22:59.642535+0000","last_fullsized":"2023-07-06T02:22:59.642535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463130+0000","last_change":"2023-07-06T02:22:57.453513+0000","last_active":"2023-07-06T02:22:58.463130+0000","last_peered":"2023-07-06T02:22:58.463130+0000","last_clean":"2023-07-06T02:22:58.463130+0000","last_became_active":"2023-07-06T02:22:57.453163+0000","last_became_peered":"2023-07-06T02:22:57.453163+0000","last_unstale":"2023-07-06T02:22:58.463130+0000","last_undegraded":"2023-07-06T02:22:58.463130+0000","last_fullsized":"2023-07-06T02:22:58.463130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:06:26.324856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545616+0000","last_change":"2023-07-06T02:22:47.613219+0000","last_active":"2023-07-06T02:22:58.545616+0000","last_peered":"2023-07-06T02:22:58.545616+0000","last_clean":"2023-07-06T02:22:58.545616+0000","last_became_active":"2023-07-06T02:22:47.612787+0000","last_became_peered":"2023-07-06T02:22:47.612787+0000","last_unstale":"2023-07-06T02:22:58.545616+0000","last_undegraded":"2023-07-06T02:22:58.545616+0000","last_fullsized":"2023-07-06T02:22:58.545616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:24.728713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496749+0000","last_change":"2023-07-06T02:22:52.421146+0000","last_active":"2023-07-06T02:22:58.496749+0000","last_peered":"2023-07-06T02:22:58.496749+0000","last_clean":"2023-07-06T02:22:58.496749+0000","last_became_active":"2023-07-06T02:22:52.420937+0000","last_became_peered":"2023-07-06T02:22:52.420937+0000","last_unstale":"2023-07-06T02:22:58.496749+0000","last_undegraded":"2023-07-06T02:22:58.496749+0000","last_fullsized":"2023-07-06T02:22:58.496749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:41.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497242+0000","last_change":"2023-07-06T02:22:53.428966+0000","last_active":"2023-07-06T02:22:58.497242+0000","last_peered":"2023-07-06T02:22:58.497242+0000","last_clean":"2023-07-06T02:22:58.497242+0000","last_became_active":"2023-07-06T02:22:53.428644+0000","last_became_peered":"2023-07-06T02:22:53.428644+0000","last_unstale":"2023-07-06T02:22:58.497242+0000","last_undegraded":"2023-07-06T02:22:58.497242+0000","last_fullsized":"2023-07-06T02:22:58.497242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:10.006953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545738+0000","last_change":"2023-07-06T02:22:57.445443+0000","last_active":"2023-07-06T02:22:58.545738+0000","last_peered":"2023-07-06T02:22:58.545738+0000","last_clean":"2023-07-06T02:22:58.545738+0000","last_became_active":"2023-07-06T02:22:57.445042+0000","last_became_peered":"2023-07-06T02:22:57.445042+0000","last_unstale":"2023-07-06T02:22:58.545738+0000","last_undegraded":"2023-07-06T02:22:58.545738+0000","last_fullsized":"2023-07-06T02:22:58.545738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:09.069048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545450+0000","last_change":"2023-07-06T02:22:47.613102+0000","last_active":"2023-07-06T02:22:58.545450+0000","last_peered":"2023-07-06T02:22:58.545450+0000","last_clean":"2023-07-06T02:22:58.545450+0000","last_became_active":"2023-07-06T02:22:47.612655+0000","last_became_peered":"2023-07-06T02:22:47.612655+0000","last_unstale":"2023-07-06T02:22:58.545450+0000","last_undegraded":"2023-07-06T02:22:58.545450+0000","last_fullsized":"2023-07-06T02:22:58.545450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:30.336091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462609+0000","last_change":"2023-07-06T02:22:52.419398+0000","last_active":"2023-07-06T02:22:58.462609+0000","last_peered":"2023-07-06T02:22:58.462609+0000","last_clean":"2023-07-06T02:22:58.462609+0000","last_became_active":"2023-07-06T02:22:52.419139+0000","last_became_peered":"2023-07-06T02:22:52.419139+0000","last_unstale":"2023-07-06T02:22:58.462609+0000","last_undegraded":"2023-07-06T02:22:58.462609+0000","last_fullsized":"2023-07-06T02:22:58.462609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463313+0000","last_change":"2023-07-06T02:22:53.425795+0000","last_active":"2023-07-06T02:22:58.463313+0000","last_peered":"2023-07-06T02:22:58.463313+0000","last_clean":"2023-07-06T02:22:58.463313+0000","last_became_active":"2023-07-06T02:22:53.425518+0000","last_became_peered":"2023-07-06T02:22:53.425518+0000","last_unstale":"2023-07-06T02:22:58.463313+0000","last_undegraded":"2023-07-06T02:22:58.463313+0000","last_fullsized":"2023-07-06T02:22:58.463313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:21:40.141734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642407+0000","last_change":"2023-07-06T02:22:57.457031+0000","last_active":"2023-07-06T02:22:59.642407+0000","last_peered":"2023-07-06T02:22:59.642407+0000","last_clean":"2023-07-06T02:22:59.642407+0000","last_became_active":"2023-07-06T02:22:57.456653+0000","last_became_peered":"2023-07-06T02:22:57.456653+0000","last_unstale":"2023-07-06T02:22:59.642407+0000","last_undegraded":"2023-07-06T02:22:59.642407+0000","last_fullsized":"2023-07-06T02:22:59.642407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:34.843149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642558+0000","last_change":"2023-07-06T02:22:47.613009+0000","last_active":"2023-07-06T02:22:59.642558+0000","last_peered":"2023-07-06T02:22:59.642558+0000","last_clean":"2023-07-06T02:22:59.642558+0000","last_became_active":"2023-07-06T02:22:47.612664+0000","last_became_peered":"2023-07-06T02:22:47.612664+0000","last_unstale":"2023-07-06T02:22:59.642558+0000","last_undegraded":"2023-07-06T02:22:59.642558+0000","last_fullsized":"2023-07-06T02:22:59.642558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:35.690988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623032+0000","last_change":"2023-07-06T02:22:52.417974+0000","last_active":"2023-07-06T02:22:58.623032+0000","last_peered":"2023-07-06T02:22:58.623032+0000","last_clean":"2023-07-06T02:22:58.623032+0000","last_became_active":"2023-07-06T02:22:52.417681+0000","last_became_peered":"2023-07-06T02:22:52.417681+0000","last_unstale":"2023-07-06T02:22:58.623032+0000","last_undegraded":"2023-07-06T02:22:58.623032+0000","last_fullsized":"2023-07-06T02:22:58.623032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:28:34.626494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496776+0000","last_change":"2023-07-06T02:22:53.424117+0000","last_active":"2023-07-06T02:22:58.496776+0000","last_peered":"2023-07-06T02:22:58.496776+0000","last_clean":"2023-07-06T02:22:58.496776+0000","last_became_active":"2023-07-06T02:22:53.423755+0000","last_became_peered":"2023-07-06T02:22:53.423755+0000","last_unstale":"2023-07-06T02:22:58.496776+0000","last_undegraded":"2023-07-06T02:22:58.496776+0000","last_fullsized":"2023-07-06T02:22:58.496776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:48.677462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463117+0000","last_change":"2023-07-06T02:22:57.451130+0000","last_active":"2023-07-06T02:22:58.463117+0000","last_peered":"2023-07-06T02:22:58.463117+0000","last_clean":"2023-07-06T02:22:58.463117+0000","last_became_active":"2023-07-06T02:22:57.450726+0000","last_became_peered":"2023-07-06T02:22:57.450726+0000","last_unstale":"2023-07-06T02:22:58.463117+0000","last_undegraded":"2023-07-06T02:22:58.463117+0000","last_fullsized":"2023-07-06T02:22:58.463117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:38.492011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463258+0000","last_change":"2023-07-06T02:22:47.612949+0000","last_active":"2023-07-06T02:22:58.463258+0000","last_peered":"2023-07-06T02:22:58.463258+0000","last_clean":"2023-07-06T02:22:58.463258+0000","last_became_active":"2023-07-06T02:22:47.612606+0000","last_became_peered":"2023-07-06T02:22:47.612606+0000","last_unstale":"2023-07-06T02:22:58.463258+0000","last_undegraded":"2023-07-06T02:22:58.463258+0000","last_fullsized":"2023-07-06T02:22:58.463258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:26:58.278350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496991+0000","last_change":"2023-07-06T02:22:52.421787+0000","last_active":"2023-07-06T02:22:58.496991+0000","last_peered":"2023-07-06T02:22:58.496991+0000","last_clean":"2023-07-06T02:22:58.496991+0000","last_became_active":"2023-07-06T02:22:52.421536+0000","last_became_peered":"2023-07-06T02:22:52.421536+0000","last_unstale":"2023-07-06T02:22:58.496991+0000","last_undegraded":"2023-07-06T02:22:58.496991+0000","last_fullsized":"2023-07-06T02:22:58.496991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:50.380161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642624+0000","last_change":"2023-07-06T02:22:53.419643+0000","last_active":"2023-07-06T02:22:59.642624+0000","last_peered":"2023-07-06T02:22:59.642624+0000","last_clean":"2023-07-06T02:22:59.642624+0000","last_became_active":"2023-07-06T02:22:53.418153+0000","last_became_peered":"2023-07-06T02:22:53.418153+0000","last_unstale":"2023-07-06T02:22:59.642624+0000","last_undegraded":"2023-07-06T02:22:59.642624+0000","last_fullsized":"2023-07-06T02:22:59.642624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642319+0000","last_change":"2023-07-06T02:22:57.455969+0000","last_active":"2023-07-06T02:22:59.642319+0000","last_peered":"2023-07-06T02:22:59.642319+0000","last_clean":"2023-07-06T02:22:59.642319+0000","last_became_active":"2023-07-06T02:22:57.455566+0000","last_became_peered":"2023-07-06T02:22:57.455566+0000","last_unstale":"2023-07-06T02:22:59.642319+0000","last_undegraded":"2023-07-06T02:22:59.642319+0000","last_fullsized":"2023-07-06T02:22:59.642319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:15.032393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463409+0000","last_change":"2023-07-06T02:22:47.329340+0000","last_active":"2023-07-06T02:22:58.463409+0000","last_peered":"2023-07-06T02:22:58.463409+0000","last_clean":"2023-07-06T02:22:58.463409+0000","last_became_active":"2023-07-06T02:22:47.329090+0000","last_became_peered":"2023-07-06T02:22:47.329090+0000","last_unstale":"2023-07-06T02:22:58.463409+0000","last_undegraded":"2023-07-06T02:22:58.463409+0000","last_fullsized":"2023-07-06T02:22:58.463409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:42.929148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643052+0000","last_change":"2023-07-06T02:22:52.422406+0000","last_active":"2023-07-06T02:22:59.643052+0000","last_peered":"2023-07-06T02:22:59.643052+0000","last_clean":"2023-07-06T02:22:59.643052+0000","last_became_active":"2023-07-06T02:22:52.422125+0000","last_became_peered":"2023-07-06T02:22:52.422125+0000","last_unstale":"2023-07-06T02:22:59.643052+0000","last_undegraded":"2023-07-06T02:22:59.643052+0000","last_fullsized":"2023-07-06T02:22:59.643052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:08.999634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496575+0000","last_change":"2023-07-06T02:22:53.428897+0000","last_active":"2023-07-06T02:22:58.496575+0000","last_peered":"2023-07-06T02:22:58.496575+0000","last_clean":"2023-07-06T02:22:58.496575+0000","last_became_active":"2023-07-06T02:22:53.428651+0000","last_became_peered":"2023-07-06T02:22:53.428651+0000","last_unstale":"2023-07-06T02:22:58.496575+0000","last_undegraded":"2023-07-06T02:22:58.496575+0000","last_fullsized":"2023-07-06T02:22:58.496575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:49.714737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497099+0000","last_change":"2023-07-06T02:22:57.454674+0000","last_active":"2023-07-06T02:22:58.497099+0000","last_peered":"2023-07-06T02:22:58.497099+0000","last_clean":"2023-07-06T02:22:58.497099+0000","last_became_active":"2023-07-06T02:22:57.454162+0000","last_became_peered":"2023-07-06T02:22:57.454162+0000","last_unstale":"2023-07-06T02:22:58.497099+0000","last_undegraded":"2023-07-06T02:22:58.497099+0000","last_fullsized":"2023-07-06T02:22:58.497099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:56.768674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642632+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:59.642632+0000","last_peered":"2023-07-06T02:22:59.642632+0000","last_clean":"2023-07-06T02:22:59.642632+0000","last_became_active":"2023-07-06T02:22:52.418794+0000","last_became_peered":"2023-07-06T02:22:52.418794+0000","last_unstale":"2023-07-06T02:22:59.642632+0000","last_undegraded":"2023-07-06T02:22:59.642632+0000","last_fullsized":"2023-07-06T02:22:59.642632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463505+0000","last_change":"2023-07-06T02:22:53.424846+0000","last_active":"2023-07-06T02:22:58.463505+0000","last_peered":"2023-07-06T02:22:58.463505+0000","last_clean":"2023-07-06T02:22:58.463505+0000","last_became_active":"2023-07-06T02:22:53.424606+0000","last_became_peered":"2023-07-06T02:22:53.424606+0000","last_unstale":"2023-07-06T02:22:58.463505+0000","last_undegraded":"2023-07-06T02:22:58.463505+0000","last_fullsized":"2023-07-06T02:22:58.463505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:31:39.592702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545637+0000","last_change":"2023-07-06T02:22:57.455280+0000","last_active":"2023-07-06T02:22:58.545637+0000","last_peered":"2023-07-06T02:22:58.545637+0000","last_clean":"2023-07-06T02:22:58.545637+0000","last_became_active":"2023-07-06T02:22:57.454860+0000","last_became_peered":"2023-07-06T02:22:57.454860+0000","last_unstale":"2023-07-06T02:22:58.545637+0000","last_undegraded":"2023-07-06T02:22:58.545637+0000","last_fullsized":"2023-07-06T02:22:58.545637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642892+0000","last_change":"2023-07-06T02:22:52.419322+0000","last_active":"2023-07-06T02:22:59.642892+0000","last_peered":"2023-07-06T02:22:59.642892+0000","last_clean":"2023-07-06T02:22:59.642892+0000","last_became_active":"2023-07-06T02:22:52.418950+0000","last_became_peered":"2023-07-06T02:22:52.418950+0000","last_unstale":"2023-07-06T02:22:59.642892+0000","last_undegraded":"2023-07-06T02:22:59.642892+0000","last_fullsized":"2023-07-06T02:22:59.642892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:22.417730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623010+0000","last_change":"2023-07-06T02:22:53.419784+0000","last_active":"2023-07-06T02:22:58.623010+0000","last_peered":"2023-07-06T02:22:58.623010+0000","last_clean":"2023-07-06T02:22:58.623010+0000","last_became_active":"2023-07-06T02:22:53.419477+0000","last_became_peered":"2023-07-06T02:22:53.419477+0000","last_unstale":"2023-07-06T02:22:58.623010+0000","last_undegraded":"2023-07-06T02:22:58.623010+0000","last_fullsized":"2023-07-06T02:22:58.623010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:07.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643049+0000","last_change":"2023-07-06T02:22:57.447138+0000","last_active":"2023-07-06T02:22:59.643049+0000","last_peered":"2023-07-06T02:22:59.643049+0000","last_clean":"2023-07-06T02:22:59.643049+0000","last_became_active":"2023-07-06T02:22:57.446786+0000","last_became_peered":"2023-07-06T02:22:57.446786+0000","last_unstale":"2023-07-06T02:22:59.643049+0000","last_undegraded":"2023-07-06T02:22:59.643049+0000","last_fullsized":"2023-07-06T02:22:59.643049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:45.545737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643069+0000","last_change":"2023-07-06T02:22:52.419733+0000","last_active":"2023-07-06T02:22:59.643069+0000","last_peered":"2023-07-06T02:22:59.643069+0000","last_clean":"2023-07-06T02:22:59.643069+0000","last_became_active":"2023-07-06T02:22:52.418972+0000","last_became_peered":"2023-07-06T02:22:52.418972+0000","last_unstale":"2023-07-06T02:22:59.643069+0000","last_undegraded":"2023-07-06T02:22:59.643069+0000","last_fullsized":"2023-07-06T02:22:59.643069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:07:57.245299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622844+0000","last_change":"2023-07-06T02:22:53.416731+0000","last_active":"2023-07-06T02:22:58.622844+0000","last_peered":"2023-07-06T02:22:58.622844+0000","last_clean":"2023-07-06T02:22:58.622844+0000","last_became_active":"2023-07-06T02:22:53.416412+0000","last_became_peered":"2023-07-06T02:22:53.416412+0000","last_unstale":"2023-07-06T02:22:58.622844+0000","last_undegraded":"2023-07-06T02:22:58.622844+0000","last_fullsized":"2023-07-06T02:22:58.622844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:36:20.268819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622716+0000","last_change":"2023-07-06T02:22:57.461025+0000","last_active":"2023-07-06T02:22:58.622716+0000","last_peered":"2023-07-06T02:22:58.622716+0000","last_clean":"2023-07-06T02:22:58.622716+0000","last_became_active":"2023-07-06T02:22:57.460633+0000","last_became_peered":"2023-07-06T02:22:57.460633+0000","last_unstale":"2023-07-06T02:22:58.622716+0000","last_undegraded":"2023-07-06T02:22:58.622716+0000","last_fullsized":"2023-07-06T02:22:58.622716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:26.873513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:52.418771+0000","last_became_peered":"2023-07-06T02:22:52.418771+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:24:21.889565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463056+0000","last_change":"2023-07-06T02:22:53.421840+0000","last_active":"2023-07-06T02:22:58.463056+0000","last_peered":"2023-07-06T02:22:58.463056+0000","last_clean":"2023-07-06T02:22:58.463056+0000","last_became_active":"2023-07-06T02:22:53.421644+0000","last_became_peered":"2023-07-06T02:22:53.421644+0000","last_unstale":"2023-07-06T02:22:58.463056+0000","last_undegraded":"2023-07-06T02:22:58.463056+0000","last_fullsized":"2023-07-06T02:22:58.463056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:42:56.076946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:57.459681+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:57.459298+0000","last_became_peered":"2023-07-06T02:22:57.459298+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:49.871715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642530+0000","last_change":"2023-07-06T02:22:52.425605+0000","last_active":"2023-07-06T02:22:59.642530+0000","last_peered":"2023-07-06T02:22:59.642530+0000","last_clean":"2023-07-06T02:22:59.642530+0000","last_became_active":"2023-07-06T02:22:52.425345+0000","last_became_peered":"2023-07-06T02:22:52.425345+0000","last_unstale":"2023-07-06T02:22:59.642530+0000","last_undegraded":"2023-07-06T02:22:59.642530+0000","last_fullsized":"2023-07-06T02:22:59.642530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622815+0000","last_change":"2023-07-06T02:22:53.422473+0000","last_active":"2023-07-06T02:22:58.622815+0000","last_peered":"2023-07-06T02:22:58.622815+0000","last_clean":"2023-07-06T02:22:58.622815+0000","last_became_active":"2023-07-06T02:22:53.422171+0000","last_became_peered":"2023-07-06T02:22:53.422171+0000","last_unstale":"2023-07-06T02:22:58.622815+0000","last_undegraded":"2023-07-06T02:22:58.622815+0000","last_fullsized":"2023-07-06T02:22:58.622815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:35.957039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462966+0000","last_change":"2023-07-06T02:22:53.420559+0000","last_active":"2023-07-06T02:22:58.462966+0000","last_peered":"2023-07-06T02:22:58.462966+0000","last_clean":"2023-07-06T02:22:58.462966+0000","last_became_active":"2023-07-06T02:22:53.420220+0000","last_became_peered":"2023-07-06T02:22:53.420220+0000","last_unstale":"2023-07-06T02:22:58.462966+0000","last_undegraded":"2023-07-06T02:22:58.462966+0000","last_fullsized":"2023-07-06T02:22:58.462966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:11.617347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462676+0000","last_change":"2023-07-06T02:22:53.424835+0000","last_active":"2023-07-06T02:22:58.462676+0000","last_peered":"2023-07-06T02:22:58.462676+0000","last_clean":"2023-07-06T02:22:58.462676+0000","last_became_active":"2023-07-06T02:22:53.424520+0000","last_became_peered":"2023-07-06T02:22:53.424520+0000","last_unstale":"2023-07-06T02:22:58.462676+0000","last_undegraded":"2023-07-06T02:22:58.462676+0000","last_fullsized":"2023-07-06T02:22:58.462676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:27.623742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623256+0000","last_change":"2023-07-06T02:22:53.420661+0000","last_active":"2023-07-06T02:22:58.623256+0000","last_peered":"2023-07-06T02:22:58.623256+0000","last_clean":"2023-07-06T02:22:58.623256+0000","last_became_active":"2023-07-06T02:22:53.420137+0000","last_became_peered":"2023-07-06T02:22:53.420137+0000","last_unstale":"2023-07-06T02:22:58.623256+0000","last_undegraded":"2023-07-06T02:22:58.623256+0000","last_fullsized":"2023-07-06T02:22:58.623256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:07:51.965833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462455+0000","last_change":"2023-07-06T02:22:53.434477+0000","last_active":"2023-07-06T02:22:58.462455+0000","last_peered":"2023-07-06T02:22:58.462455+0000","last_clean":"2023-07-06T02:22:58.462455+0000","last_became_active":"2023-07-06T02:22:53.434127+0000","last_became_peered":"2023-07-06T02:22:53.434127+0000","last_unstale":"2023-07-06T02:22:58.462455+0000","last_undegraded":"2023-07-06T02:22:58.462455+0000","last_fullsized":"2023-07-06T02:22:58.462455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:53.433904+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:53.433683+0000","last_became_peered":"2023-07-06T02:22:53.433683+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T02:23:17.969 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T02:23:18.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.115+0000 7f5b8bdcc700 1 Processor -- start 2023-07-06T02:23:18.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.116+0000 7f5b8bdcc700 1 -- start start 2023-07-06T02:23:18.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.116+0000 7f5b8bdcc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5b8415a3e0 0x7f5b840caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.116+0000 7f5b8bdcc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8406bcd0 0x7f5b84070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.116+0000 7f5b8bdcc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8406a500 con 0x7f5b84071460 2023-07-06T02:23:18.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.116+0000 7f5b8bdcc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b840718e0 con 0x7f5b8415a3e0 2023-07-06T02:23:18.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.116+0000 7f5b8bdcc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8406ec60 con 0x7f5b8406bcd0 2023-07-06T02:23:18.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b8a369700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5b84071460 0x7f5b840c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57848/0 (socket says 172.21.15.89:57848) 2023-07-06T02:23:18.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b8a369700 1 -- 172.21.15.89:0/1851717747 learned_addr learned my addr 172.21.15.89:0/1851717747 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:18.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b89367700 1 --2- 172.21.15.89:0/1851717747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8406bcd0 0x7f5b84070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b89b68700 1 --2- 172.21.15.89:0/1851717747 >> v2:172.21.15.172:3300/0 conn(0x7f5b8415a3e0 0x7f5b840caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b88b66700 1 -- 172.21.15.89:0/1851717747 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650591454 0 0) 0x7f5b8406a500 con 0x7f5b84071460 2023-07-06T02:23:18.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b88b66700 1 -- 172.21.15.89:0/1851717747 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b700035b0 con 0x7f5b84071460 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b89367700 1 -- 172.21.15.89:0/1851717747 >> v2:172.21.15.172:3300/0 conn(0x7f5b8415a3e0 msgr2=0x7f5b840caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b89367700 1 --2- 172.21.15.89:0/1851717747 >> v2:172.21.15.172:3300/0 conn(0x7f5b8415a3e0 0x7f5b840caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b89367700 1 -- 172.21.15.89:0/1851717747 >> v1:172.21.15.89:6789/0 conn(0x7f5b84071460 legacy=0x7f5b840c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b89367700 1 -- 172.21.15.89:0/1851717747 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8406f190 con 0x7f5b8406bcd0 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b89367700 1 --2- 172.21.15.89:0/1851717747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8406bcd0 0x7f5b84070f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009840 tx=0x7f5b80009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdcab5b617a1fc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b88b66700 1 -- 172.21.15.89:0/1851717747 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80011830 con 0x7f5b8406bcd0 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b88b66700 1 -- 172.21.15.89:0/1851717747 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80011990 con 0x7f5b8406bcd0 2023-07-06T02:23:18.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.117+0000 7f5b88b66700 1 -- 172.21.15.89:0/1851717747 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80011d20 con 0x7f5b8406bcd0 2023-07-06T02:23:18.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1851717747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8406bcd0 msgr2=0x7f5b84070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1851717747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8406bcd0 0x7f5b84070f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009840 tx=0x7f5b80009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:18.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1851717747 shutdown_connections 2023-07-06T02:23:18.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1851717747 >> v2:172.21.15.172:3300/0 conn(0x7f5b8415a3e0 0x7f5b840caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1851717747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8406bcd0 0x7f5b84070f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1851717747 >> 172.21.15.89:0/1851717747 conn(0x7f5b840cf1c0 msgr2=0x7f5b840cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:18.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1851717747 shutdown_connections 2023-07-06T02:23:18.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.118+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1851717747 wait complete. 2023-07-06T02:23:18.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.119+0000 7f5b8bdcc700 1 Processor -- start 2023-07-06T02:23:18.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.119+0000 7f5b8bdcc700 1 -- start start 2023-07-06T02:23:18.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b8bdcc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8406bcd0 0x7f5b84158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b8bdcc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b84071460 0x7f5b840c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b8bdcc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8415a3e0 0x7f5b841579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b8bdcc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840c5bb0 con 0x7f5b8406bcd0 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b8bdcc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84157f20 con 0x7f5b84071460 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b8bdcc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8415a950 con 0x7f5b8415a3e0 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8406bcd0 0x7f5b84158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8406bcd0 0x7f5b84158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32948/0 (socket says 172.21.15.89:32948) 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 -- 172.21.15.89:0/1126435663 learned_addr learned my addr 172.21.15.89:0/1126435663 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b8a369700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8415a3e0 0x7f5b841579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89367700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b84071460 0x7f5b840c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 -- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8415a3e0 msgr2=0x7f5b841579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8415a3e0 0x7f5b841579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 -- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b84071460 msgr2=0x7f5b840c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b84071460 0x7f5b840c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.120+0000 7f5b89b68700 1 -- 172.21.15.89:0/1126435663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800094a0 con 0x7f5b8406bcd0 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.121+0000 7f5b89b68700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8406bcd0 0x7f5b84158100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e870 tx=0x7f5b7400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.121+0000 7f5b7a7fc700 1 -- 172.21.15.89:0/1126435663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74009590 con 0x7f5b8406bcd0 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.121+0000 7f5b7a7fc700 1 -- 172.21.15.89:0/1126435663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400ee70 con 0x7f5b8406bcd0 2023-07-06T02:23:18.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.121+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8415ac30 con 0x7f5b8406bcd0 2023-07-06T02:23:18.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.121+0000 7f5b7a7fc700 1 -- 172.21.15.89:0/1126435663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400ca50 con 0x7f5b8406bcd0 2023-07-06T02:23:18.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.121+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8415b1f0 con 0x7f5b8406bcd0 2023-07-06T02:23:18.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.122+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b84074270 con 0x7f5b8406bcd0 2023-07-06T02:23:18.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.122+0000 7f5b7a7fc700 1 -- 172.21.15.89:0/1126435663 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5b7400cbb0 con 0x7f5b8406bcd0 2023-07-06T02:23:18.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.122+0000 7f5b7a7fc700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b6806ad10 0x7f5b6806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.122+0000 7f5b7a7fc700 1 -- 172.21.15.89:0/1126435663 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5b74089740 con 0x7f5b8406bcd0 2023-07-06T02:23:18.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.122+0000 7f5b89367700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b6806ad10 0x7f5b6806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.123+0000 7f5b89367700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b6806ad10 0x7f5b6806d1c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b800090b0 tx=0x7f5b80008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.127+0000 7f5b7a7fc700 1 -- 172.21.15.89:0/1126435663 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5b74057f90 con 0x7f5b8406bcd0 2023-07-06T02:23:18.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.312+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 --> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b840cb980 con 0x7f5b6806ad10 2023-07-06T02:23:18.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.322+0000 7f5b7a7fc700 1 -- 172.21.15.89:0/1126435663 <== mgr.4100 v2:172.21.15.89:6832/3591115711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367548 (secure 0 0 0) 0x7f5b80011b00 con 0x7f5b6806ad10 2023-07-06T02:23:18.328 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:18.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b6806ad10 msgr2=0x7f5b6806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b6806ad10 0x7f5b6806d1c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b800090b0 tx=0x7f5b80008030 comp rx=0 tx=0).stop 2023-07-06T02:23:18.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8406bcd0 msgr2=0x7f5b84158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8406bcd0 0x7f5b84158100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e870 tx=0x7f5b7400ec30 comp rx=0 tx=0).stop 2023-07-06T02:23:18.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 shutdown_connections 2023-07-06T02:23:18.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b8415a3e0 0x7f5b841579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b6806ad10 0x7f5b6806d1c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b8406bcd0 0x7f5b84158100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 --2- 172.21.15.89:0/1126435663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b84071460 0x7f5b840c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 >> 172.21.15.89:0/1126435663 conn(0x7f5b840cf1c0 msgr2=0x7f5b840b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 shutdown_connections 2023-07-06T02:23:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.325+0000 7f5b8bdcc700 1 -- 172.21.15.89:0/1126435663 wait complete. 2023-07-06T02:23:18.332 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-07-06T02:23:18.340 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-07-06T02:23:17.986274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218820,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755900,"statfs":{"total":773094113280,"available":772870041600,"internally_reserved":0,"allocated":3280896,"data_stored":2026399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006166"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462978+0000","last_change":"2023-07-06T02:22:53.433054+0000","last_active":"2023-07-06T02:22:58.462978+0000","last_peered":"2023-07-06T02:22:58.462978+0000","last_clean":"2023-07-06T02:22:58.462978+0000","last_became_active":"2023-07-06T02:22:53.432749+0000","last_became_peered":"2023-07-06T02:22:53.432749+0000","last_unstale":"2023-07-06T02:22:58.462978+0000","last_undegraded":"2023-07-06T02:22:58.462978+0000","last_fullsized":"2023-07-06T02:22:58.462978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:42.597214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643097+0000","last_change":"2023-07-06T02:22:53.425731+0000","last_active":"2023-07-06T02:22:59.643097+0000","last_peered":"2023-07-06T02:22:59.643097+0000","last_clean":"2023-07-06T02:22:59.643097+0000","last_became_active":"2023-07-06T02:22:53.425403+0000","last_became_peered":"2023-07-06T02:22:53.425403+0000","last_unstale":"2023-07-06T02:22:59.643097+0000","last_undegraded":"2023-07-06T02:22:59.643097+0000","last_fullsized":"2023-07-06T02:22:59.643097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:00.409252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623835+0000","last_change":"2023-07-06T02:22:53.413131+0000","last_active":"2023-07-06T02:22:58.623835+0000","last_peered":"2023-07-06T02:22:58.623835+0000","last_clean":"2023-07-06T02:22:58.623835+0000","last_became_active":"2023-07-06T02:22:53.412839+0000","last_became_peered":"2023-07-06T02:22:53.412839+0000","last_unstale":"2023-07-06T02:22:58.623835+0000","last_undegraded":"2023-07-06T02:22:58.623835+0000","last_fullsized":"2023-07-06T02:22:58.623835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:51.080641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463166+0000","last_change":"2023-07-06T02:22:53.420056+0000","last_active":"2023-07-06T02:22:58.463166+0000","last_peered":"2023-07-06T02:22:58.463166+0000","last_clean":"2023-07-06T02:22:58.463166+0000","last_became_active":"2023-07-06T02:22:53.419779+0000","last_became_peered":"2023-07-06T02:22:53.419779+0000","last_unstale":"2023-07-06T02:22:58.463166+0000","last_undegraded":"2023-07-06T02:22:58.463166+0000","last_fullsized":"2023-07-06T02:22:58.463166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:37.102968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642705+0000","last_change":"2023-07-06T02:22:53.420867+0000","last_active":"2023-07-06T02:22:59.642705+0000","last_peered":"2023-07-06T02:22:59.642705+0000","last_clean":"2023-07-06T02:22:59.642705+0000","last_became_active":"2023-07-06T02:22:53.419716+0000","last_became_peered":"2023-07-06T02:22:53.419716+0000","last_unstale":"2023-07-06T02:22:59.642705+0000","last_undegraded":"2023-07-06T02:22:59.642705+0000","last_fullsized":"2023-07-06T02:22:59.642705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:33.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623263+0000","last_change":"2023-07-06T02:22:53.412521+0000","last_active":"2023-07-06T02:22:58.623263+0000","last_peered":"2023-07-06T02:22:58.623263+0000","last_clean":"2023-07-06T02:22:58.623263+0000","last_became_active":"2023-07-06T02:22:53.412230+0000","last_became_peered":"2023-07-06T02:22:53.412230+0000","last_unstale":"2023-07-06T02:22:58.623263+0000","last_undegraded":"2023-07-06T02:22:58.623263+0000","last_fullsized":"2023-07-06T02:22:58.623263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:44.353524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545634+0000","last_change":"2023-07-06T02:22:53.427954+0000","last_active":"2023-07-06T02:22:58.545634+0000","last_peered":"2023-07-06T02:22:58.545634+0000","last_clean":"2023-07-06T02:22:58.545634+0000","last_became_active":"2023-07-06T02:22:53.426315+0000","last_became_peered":"2023-07-06T02:22:53.426315+0000","last_unstale":"2023-07-06T02:22:58.545634+0000","last_undegraded":"2023-07-06T02:22:58.545634+0000","last_fullsized":"2023-07-06T02:22:58.545634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642615+0000","last_change":"2023-07-06T02:22:53.425882+0000","last_active":"2023-07-06T02:22:59.642615+0000","last_peered":"2023-07-06T02:22:59.642615+0000","last_clean":"2023-07-06T02:22:59.642615+0000","last_became_active":"2023-07-06T02:22:53.425583+0000","last_became_peered":"2023-07-06T02:22:53.425583+0000","last_unstale":"2023-07-06T02:22:59.642615+0000","last_undegraded":"2023-07-06T02:22:59.642615+0000","last_fullsized":"2023-07-06T02:22:59.642615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:51.387481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642268+0000","last_change":"2023-07-06T02:22:53.427106+0000","last_active":"2023-07-06T02:22:59.642268+0000","last_peered":"2023-07-06T02:22:59.642268+0000","last_clean":"2023-07-06T02:22:59.642268+0000","last_became_active":"2023-07-06T02:22:53.426852+0000","last_became_peered":"2023-07-06T02:22:53.426852+0000","last_unstale":"2023-07-06T02:22:59.642268+0000","last_undegraded":"2023-07-06T02:22:59.642268+0000","last_fullsized":"2023-07-06T02:22:59.642268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643507+0000","last_change":"2023-07-06T02:22:53.421592+0000","last_active":"2023-07-06T02:22:59.643507+0000","last_peered":"2023-07-06T02:22:59.643507+0000","last_clean":"2023-07-06T02:22:59.643507+0000","last_became_active":"2023-07-06T02:22:53.421245+0000","last_became_peered":"2023-07-06T02:22:53.421245+0000","last_unstale":"2023-07-06T02:22:59.643507+0000","last_undegraded":"2023-07-06T02:22:59.643507+0000","last_fullsized":"2023-07-06T02:22:59.643507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623746+0000","last_change":"2023-07-06T02:22:53.424792+0000","last_active":"2023-07-06T02:22:58.623746+0000","last_peered":"2023-07-06T02:22:58.623746+0000","last_clean":"2023-07-06T02:22:58.623746+0000","last_became_active":"2023-07-06T02:22:53.424498+0000","last_became_peered":"2023-07-06T02:22:53.424498+0000","last_unstale":"2023-07-06T02:22:58.623746+0000","last_undegraded":"2023-07-06T02:22:58.623746+0000","last_fullsized":"2023-07-06T02:22:58.623746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:55:40.483266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462649+0000","last_change":"2023-07-06T02:22:53.424739+0000","last_active":"2023-07-06T02:22:58.462649+0000","last_peered":"2023-07-06T02:22:58.462649+0000","last_clean":"2023-07-06T02:22:58.462649+0000","last_became_active":"2023-07-06T02:22:53.424382+0000","last_became_peered":"2023-07-06T02:22:53.424382+0000","last_unstale":"2023-07-06T02:22:58.462649+0000","last_undegraded":"2023-07-06T02:22:58.462649+0000","last_fullsized":"2023-07-06T02:22:58.462649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:08:50.637616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462913+0000","last_change":"2023-07-06T02:22:53.435357+0000","last_active":"2023-07-06T02:22:58.462913+0000","last_peered":"2023-07-06T02:22:58.462913+0000","last_clean":"2023-07-06T02:22:58.462913+0000","last_became_active":"2023-07-06T02:22:53.435068+0000","last_became_peered":"2023-07-06T02:22:53.435068+0000","last_unstale":"2023-07-06T02:22:58.462913+0000","last_undegraded":"2023-07-06T02:22:58.462913+0000","last_fullsized":"2023-07-06T02:22:58.462913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:05.764438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642840+0000","last_change":"2023-07-06T02:22:53.419307+0000","last_active":"2023-07-06T02:22:59.642840+0000","last_peered":"2023-07-06T02:22:59.642840+0000","last_clean":"2023-07-06T02:22:59.642840+0000","last_became_active":"2023-07-06T02:22:53.419010+0000","last_became_peered":"2023-07-06T02:22:53.419010+0000","last_unstale":"2023-07-06T02:22:59.642840+0000","last_undegraded":"2023-07-06T02:22:59.642840+0000","last_fullsized":"2023-07-06T02:22:59.642840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:52.740513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545431+0000","last_change":"2023-07-06T02:22:53.427932+0000","last_active":"2023-07-06T02:22:58.545431+0000","last_peered":"2023-07-06T02:22:58.545431+0000","last_clean":"2023-07-06T02:22:58.545431+0000","last_became_active":"2023-07-06T02:22:53.426280+0000","last_became_peered":"2023-07-06T02:22:53.426280+0000","last_unstale":"2023-07-06T02:22:58.545431+0000","last_undegraded":"2023-07-06T02:22:58.545431+0000","last_fullsized":"2023-07-06T02:22:58.545431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:05.507588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622966+0000","last_change":"2023-07-06T02:22:53.412012+0000","last_active":"2023-07-06T02:22:58.622966+0000","last_peered":"2023-07-06T02:22:58.622966+0000","last_clean":"2023-07-06T02:22:58.622966+0000","last_became_active":"2023-07-06T02:22:53.411675+0000","last_became_peered":"2023-07-06T02:22:53.411675+0000","last_unstale":"2023-07-06T02:22:58.622966+0000","last_undegraded":"2023-07-06T02:22:58.622966+0000","last_fullsized":"2023-07-06T02:22:58.622966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:13:54.565351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:53.432526+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:53.432240+0000","last_became_peered":"2023-07-06T02:22:53.432240+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:52.467944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623666+0000","last_change":"2023-07-06T02:22:53.419854+0000","last_active":"2023-07-06T02:22:58.623666+0000","last_peered":"2023-07-06T02:22:58.623666+0000","last_clean":"2023-07-06T02:22:58.623666+0000","last_became_active":"2023-07-06T02:22:53.419569+0000","last_became_peered":"2023-07-06T02:22:53.419569+0000","last_unstale":"2023-07-06T02:22:58.623666+0000","last_undegraded":"2023-07-06T02:22:58.623666+0000","last_fullsized":"2023-07-06T02:22:58.623666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:44:47.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463157+0000","last_change":"2023-07-06T02:22:53.422202+0000","last_active":"2023-07-06T02:22:58.463157+0000","last_peered":"2023-07-06T02:22:58.463157+0000","last_clean":"2023-07-06T02:22:58.463157+0000","last_became_active":"2023-07-06T02:22:53.421981+0000","last_became_peered":"2023-07-06T02:22:53.421981+0000","last_unstale":"2023-07-06T02:22:58.463157+0000","last_undegraded":"2023-07-06T02:22:58.463157+0000","last_fullsized":"2023-07-06T02:22:58.463157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:55.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623252+0000","last_change":"2023-07-06T02:22:53.421294+0000","last_active":"2023-07-06T02:22:58.623252+0000","last_peered":"2023-07-06T02:22:58.623252+0000","last_clean":"2023-07-06T02:22:58.623252+0000","last_became_active":"2023-07-06T02:22:53.420987+0000","last_became_peered":"2023-07-06T02:22:53.420987+0000","last_unstale":"2023-07-06T02:22:58.623252+0000","last_undegraded":"2023-07-06T02:22:58.623252+0000","last_fullsized":"2023-07-06T02:22:58.623252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:07:23.523735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545995+0000","last_change":"2023-07-06T02:22:53.423456+0000","last_active":"2023-07-06T02:22:58.545995+0000","last_peered":"2023-07-06T02:22:58.545995+0000","last_clean":"2023-07-06T02:22:58.545995+0000","last_became_active":"2023-07-06T02:22:53.423131+0000","last_became_peered":"2023-07-06T02:22:53.423131+0000","last_unstale":"2023-07-06T02:22:58.545995+0000","last_undegraded":"2023-07-06T02:22:58.545995+0000","last_fullsized":"2023-07-06T02:22:58.545995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641976+0000","last_change":"2023-07-06T02:22:53.424047+0000","last_active":"2023-07-06T02:22:59.641976+0000","last_peered":"2023-07-06T02:22:59.641976+0000","last_clean":"2023-07-06T02:22:59.641976+0000","last_became_active":"2023-07-06T02:22:53.423692+0000","last_became_peered":"2023-07-06T02:22:53.423692+0000","last_unstale":"2023-07-06T02:22:59.641976+0000","last_undegraded":"2023-07-06T02:22:59.641976+0000","last_fullsized":"2023-07-06T02:22:59.641976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.432531+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.432245+0000","last_became_peered":"2023-07-06T02:22:53.432245+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:09:56.578120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545462+0000","last_change":"2023-07-06T02:22:53.426016+0000","last_active":"2023-07-06T02:22:58.545462+0000","last_peered":"2023-07-06T02:22:58.545462+0000","last_clean":"2023-07-06T02:22:58.545462+0000","last_became_active":"2023-07-06T02:22:53.417030+0000","last_became_peered":"2023-07-06T02:22:53.417030+0000","last_unstale":"2023-07-06T02:22:58.545462+0000","last_undegraded":"2023-07-06T02:22:58.545462+0000","last_fullsized":"2023-07-06T02:22:58.545462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497209+0000","last_change":"2023-07-06T02:22:53.422907+0000","last_active":"2023-07-06T02:22:58.497209+0000","last_peered":"2023-07-06T02:22:58.497209+0000","last_clean":"2023-07-06T02:22:58.497209+0000","last_became_active":"2023-07-06T02:22:53.422617+0000","last_became_peered":"2023-07-06T02:22:53.422617+0000","last_unstale":"2023-07-06T02:22:58.497209+0000","last_undegraded":"2023-07-06T02:22:58.497209+0000","last_fullsized":"2023-07-06T02:22:58.497209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:41.517576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497094+0000","last_change":"2023-07-06T02:22:53.422413+0000","last_active":"2023-07-06T02:22:58.497094+0000","last_peered":"2023-07-06T02:22:58.497094+0000","last_clean":"2023-07-06T02:22:58.497094+0000","last_became_active":"2023-07-06T02:22:53.422025+0000","last_became_peered":"2023-07-06T02:22:53.422025+0000","last_unstale":"2023-07-06T02:22:58.497094+0000","last_undegraded":"2023-07-06T02:22:58.497094+0000","last_fullsized":"2023-07-06T02:22:58.497094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:23.425050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643229+0000","last_change":"2023-07-06T02:22:53.434688+0000","last_active":"2023-07-06T02:22:59.643229+0000","last_peered":"2023-07-06T02:22:59.643229+0000","last_clean":"2023-07-06T02:22:59.643229+0000","last_became_active":"2023-07-06T02:22:53.434423+0000","last_became_peered":"2023-07-06T02:22:53.434423+0000","last_unstale":"2023-07-06T02:22:59.643229+0000","last_undegraded":"2023-07-06T02:22:59.643229+0000","last_fullsized":"2023-07-06T02:22:59.643229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:11.004232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623583+0000","last_change":"2023-07-06T02:22:57.463283+0000","last_active":"2023-07-06T02:22:58.623583+0000","last_peered":"2023-07-06T02:22:58.623583+0000","last_clean":"2023-07-06T02:22:58.623583+0000","last_became_active":"2023-07-06T02:22:57.462913+0000","last_became_peered":"2023-07-06T02:22:57.462913+0000","last_unstale":"2023-07-06T02:22:58.623583+0000","last_undegraded":"2023-07-06T02:22:58.623583+0000","last_fullsized":"2023-07-06T02:22:58.623583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:29.980455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623061+0000","last_change":"2023-07-06T02:22:52.423689+0000","last_active":"2023-07-06T02:22:58.623061+0000","last_peered":"2023-07-06T02:22:58.623061+0000","last_clean":"2023-07-06T02:22:58.623061+0000","last_became_active":"2023-07-06T02:22:52.423464+0000","last_became_peered":"2023-07-06T02:22:52.423464+0000","last_unstale":"2023-07-06T02:22:58.623061+0000","last_undegraded":"2023-07-06T02:22:58.623061+0000","last_fullsized":"2023-07-06T02:22:58.623061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:34.698538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463653+0000","last_change":"2023-07-06T02:22:53.426231+0000","last_active":"2023-07-06T02:22:58.463653+0000","last_peered":"2023-07-06T02:22:58.463653+0000","last_clean":"2023-07-06T02:22:58.463653+0000","last_became_active":"2023-07-06T02:22:53.425923+0000","last_became_peered":"2023-07-06T02:22:53.425923+0000","last_unstale":"2023-07-06T02:22:58.463653+0000","last_undegraded":"2023-07-06T02:22:58.463653+0000","last_fullsized":"2023-07-06T02:22:58.463653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:22.845352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463216+0000","last_change":"2023-07-06T02:22:57.461661+0000","last_active":"2023-07-06T02:22:58.463216+0000","last_peered":"2023-07-06T02:22:58.463216+0000","last_clean":"2023-07-06T02:22:58.463216+0000","last_became_active":"2023-07-06T02:22:57.461262+0000","last_became_peered":"2023-07-06T02:22:57.461262+0000","last_unstale":"2023-07-06T02:22:58.463216+0000","last_undegraded":"2023-07-06T02:22:58.463216+0000","last_fullsized":"2023-07-06T02:22:58.463216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:05.846802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496610+0000","last_change":"2023-07-06T02:22:52.424261+0000","last_active":"2023-07-06T02:22:58.496610+0000","last_peered":"2023-07-06T02:22:58.496610+0000","last_clean":"2023-07-06T02:22:58.496610+0000","last_became_active":"2023-07-06T02:22:52.423984+0000","last_became_peered":"2023-07-06T02:22:52.423984+0000","last_unstale":"2023-07-06T02:22:58.496610+0000","last_undegraded":"2023-07-06T02:22:58.496610+0000","last_fullsized":"2023-07-06T02:22:58.496610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:56.520485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642313+0000","last_change":"2023-07-06T02:22:53.417163+0000","last_active":"2023-07-06T02:22:59.642313+0000","last_peered":"2023-07-06T02:22:59.642313+0000","last_clean":"2023-07-06T02:22:59.642313+0000","last_became_active":"2023-07-06T02:22:53.416875+0000","last_became_peered":"2023-07-06T02:22:53.416875+0000","last_unstale":"2023-07-06T02:22:59.642313+0000","last_undegraded":"2023-07-06T02:22:59.642313+0000","last_fullsized":"2023-07-06T02:22:59.642313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642441+0000","last_change":"2023-07-06T02:22:57.455750+0000","last_active":"2023-07-06T02:22:59.642441+0000","last_peered":"2023-07-06T02:22:59.642441+0000","last_clean":"2023-07-06T02:22:59.642441+0000","last_became_active":"2023-07-06T02:22:57.455331+0000","last_became_peered":"2023-07-06T02:22:57.455331+0000","last_unstale":"2023-07-06T02:22:59.642441+0000","last_undegraded":"2023-07-06T02:22:59.642441+0000","last_fullsized":"2023-07-06T02:22:59.642441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462903+0000","last_change":"2023-07-06T02:22:52.416577+0000","last_active":"2023-07-06T02:22:58.462903+0000","last_peered":"2023-07-06T02:22:58.462903+0000","last_clean":"2023-07-06T02:22:58.462903+0000","last_became_active":"2023-07-06T02:22:52.416144+0000","last_became_peered":"2023-07-06T02:22:52.416144+0000","last_unstale":"2023-07-06T02:22:58.462903+0000","last_undegraded":"2023-07-06T02:22:58.462903+0000","last_fullsized":"2023-07-06T02:22:58.462903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:03:22.897984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463486+0000","last_change":"2023-07-06T02:22:53.429089+0000","last_active":"2023-07-06T02:22:58.463486+0000","last_peered":"2023-07-06T02:22:58.463486+0000","last_clean":"2023-07-06T02:22:58.463486+0000","last_became_active":"2023-07-06T02:22:53.428809+0000","last_became_peered":"2023-07-06T02:22:53.428809+0000","last_unstale":"2023-07-06T02:22:58.463486+0000","last_undegraded":"2023-07-06T02:22:58.463486+0000","last_fullsized":"2023-07-06T02:22:58.463486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:30:16.217797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462711+0000","last_change":"2023-07-06T02:22:57.455092+0000","last_active":"2023-07-06T02:22:58.462711+0000","last_peered":"2023-07-06T02:22:58.462711+0000","last_clean":"2023-07-06T02:22:58.462711+0000","last_became_active":"2023-07-06T02:22:57.454707+0000","last_became_peered":"2023-07-06T02:22:57.454707+0000","last_unstale":"2023-07-06T02:22:58.462711+0000","last_undegraded":"2023-07-06T02:22:58.462711+0000","last_fullsized":"2023-07-06T02:22:58.462711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:09:16.226734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642383+0000","last_change":"2023-07-06T02:22:52.428104+0000","last_active":"2023-07-06T02:22:59.642383+0000","last_peered":"2023-07-06T02:22:59.642383+0000","last_clean":"2023-07-06T02:22:59.642383+0000","last_became_active":"2023-07-06T02:22:52.427863+0000","last_became_peered":"2023-07-06T02:22:52.427863+0000","last_unstale":"2023-07-06T02:22:59.642383+0000","last_undegraded":"2023-07-06T02:22:59.642383+0000","last_fullsized":"2023-07-06T02:22:59.642383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:52.064862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462778+0000","last_change":"2023-07-06T02:22:53.424797+0000","last_active":"2023-07-06T02:22:58.462778+0000","last_peered":"2023-07-06T02:22:58.462778+0000","last_clean":"2023-07-06T02:22:58.462778+0000","last_became_active":"2023-07-06T02:22:53.424474+0000","last_became_peered":"2023-07-06T02:22:53.424474+0000","last_unstale":"2023-07-06T02:22:58.462778+0000","last_undegraded":"2023-07-06T02:22:58.462778+0000","last_fullsized":"2023-07-06T02:22:58.462778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:31:13.843201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545356+0000","last_change":"2023-07-06T02:22:57.457309+0000","last_active":"2023-07-06T02:22:58.545356+0000","last_peered":"2023-07-06T02:22:58.545356+0000","last_clean":"2023-07-06T02:22:58.545356+0000","last_became_active":"2023-07-06T02:22:57.457008+0000","last_became_peered":"2023-07-06T02:22:57.457008+0000","last_unstale":"2023-07-06T02:22:58.545356+0000","last_undegraded":"2023-07-06T02:22:58.545356+0000","last_fullsized":"2023-07-06T02:22:58.545356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:04.437413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643326+0000","last_change":"2023-07-06T02:22:52.423048+0000","last_active":"2023-07-06T02:22:59.643326+0000","last_peered":"2023-07-06T02:22:59.643326+0000","last_clean":"2023-07-06T02:22:59.643326+0000","last_became_active":"2023-07-06T02:22:52.422801+0000","last_became_peered":"2023-07-06T02:22:52.422801+0000","last_unstale":"2023-07-06T02:22:59.643326+0000","last_undegraded":"2023-07-06T02:22:59.643326+0000","last_fullsized":"2023-07-06T02:22:59.643326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:29.807668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462478+0000","last_change":"2023-07-06T02:22:53.434004+0000","last_active":"2023-07-06T02:22:58.462478+0000","last_peered":"2023-07-06T02:22:58.462478+0000","last_clean":"2023-07-06T02:22:58.462478+0000","last_became_active":"2023-07-06T02:22:53.433739+0000","last_became_peered":"2023-07-06T02:22:53.433739+0000","last_unstale":"2023-07-06T02:22:58.462478+0000","last_undegraded":"2023-07-06T02:22:58.462478+0000","last_fullsized":"2023-07-06T02:22:58.462478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463415+0000","last_change":"2023-07-06T02:22:57.452160+0000","last_active":"2023-07-06T02:22:58.463415+0000","last_peered":"2023-07-06T02:22:58.463415+0000","last_clean":"2023-07-06T02:22:58.463415+0000","last_became_active":"2023-07-06T02:22:57.451507+0000","last_became_peered":"2023-07-06T02:22:57.451507+0000","last_unstale":"2023-07-06T02:22:58.463415+0000","last_undegraded":"2023-07-06T02:22:58.463415+0000","last_fullsized":"2023-07-06T02:22:58.463415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:10:18.060753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546029+0000","last_change":"2023-07-06T02:22:52.419096+0000","last_active":"2023-07-06T02:22:58.546029+0000","last_peered":"2023-07-06T02:22:58.546029+0000","last_clean":"2023-07-06T02:22:58.546029+0000","last_became_active":"2023-07-06T02:22:52.418844+0000","last_became_peered":"2023-07-06T02:22:52.418844+0000","last_unstale":"2023-07-06T02:22:58.546029+0000","last_undegraded":"2023-07-06T02:22:58.546029+0000","last_fullsized":"2023-07-06T02:22:58.546029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:14.974840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496797+0000","last_change":"2023-07-06T02:22:53.425547+0000","last_active":"2023-07-06T02:22:58.496797+0000","last_peered":"2023-07-06T02:22:58.496797+0000","last_clean":"2023-07-06T02:22:58.496797+0000","last_became_active":"2023-07-06T02:22:53.425239+0000","last_became_peered":"2023-07-06T02:22:53.425239+0000","last_unstale":"2023-07-06T02:22:58.496797+0000","last_undegraded":"2023-07-06T02:22:58.496797+0000","last_fullsized":"2023-07-06T02:22:58.496797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:32.244431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496944+0000","last_change":"2023-07-06T02:22:57.453836+0000","last_active":"2023-07-06T02:22:58.496944+0000","last_peered":"2023-07-06T02:22:58.496944+0000","last_clean":"2023-07-06T02:22:58.496944+0000","last_became_active":"2023-07-06T02:22:57.451637+0000","last_became_peered":"2023-07-06T02:22:57.451637+0000","last_unstale":"2023-07-06T02:22:58.496944+0000","last_undegraded":"2023-07-06T02:22:58.496944+0000","last_fullsized":"2023-07-06T02:22:58.496944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:45.805424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:23:03.321857+0000","last_change":"2023-07-06T02:22:52.421021+0000","last_active":"2023-07-06T02:23:03.321857+0000","last_peered":"2023-07-06T02:23:03.321857+0000","last_clean":"2023-07-06T02:23:03.321857+0000","last_became_active":"2023-07-06T02:22:52.420768+0000","last_became_peered":"2023-07-06T02:22:52.420768+0000","last_unstale":"2023-07-06T02:23:03.321857+0000","last_undegraded":"2023-07-06T02:23:03.321857+0000","last_fullsized":"2023-07-06T02:23:03.321857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:10:06.879321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623414+0000","last_change":"2023-07-06T02:22:53.420729+0000","last_active":"2023-07-06T02:22:58.623414+0000","last_peered":"2023-07-06T02:22:58.623414+0000","last_clean":"2023-07-06T02:22:58.623414+0000","last_became_active":"2023-07-06T02:22:53.418685+0000","last_became_peered":"2023-07-06T02:22:53.418685+0000","last_unstale":"2023-07-06T02:22:58.623414+0000","last_undegraded":"2023-07-06T02:22:58.623414+0000","last_fullsized":"2023-07-06T02:22:58.623414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:47.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643458+0000","last_change":"2023-07-06T02:22:57.455310+0000","last_active":"2023-07-06T02:22:59.643458+0000","last_peered":"2023-07-06T02:22:59.643458+0000","last_clean":"2023-07-06T02:22:59.643458+0000","last_became_active":"2023-07-06T02:22:57.454518+0000","last_became_peered":"2023-07-06T02:22:57.454518+0000","last_unstale":"2023-07-06T02:22:59.643458+0000","last_undegraded":"2023-07-06T02:22:59.643458+0000","last_fullsized":"2023-07-06T02:22:59.643458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:57.100343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545849+0000","last_change":"2023-07-06T02:22:52.421399+0000","last_active":"2023-07-06T02:22:58.545849+0000","last_peered":"2023-07-06T02:22:58.545849+0000","last_clean":"2023-07-06T02:22:58.545849+0000","last_became_active":"2023-07-06T02:22:52.420062+0000","last_became_peered":"2023-07-06T02:22:52.420062+0000","last_unstale":"2023-07-06T02:22:58.545849+0000","last_undegraded":"2023-07-06T02:22:58.545849+0000","last_fullsized":"2023-07-06T02:22:58.545849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:53.425567+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:53.425272+0000","last_became_peered":"2023-07-06T02:22:53.425272+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:45.959463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463853+0000","last_change":"2023-07-06T02:22:57.452727+0000","last_active":"2023-07-06T02:22:58.463853+0000","last_peered":"2023-07-06T02:22:58.463853+0000","last_clean":"2023-07-06T02:22:58.463853+0000","last_became_active":"2023-07-06T02:22:57.452043+0000","last_became_peered":"2023-07-06T02:22:57.452043+0000","last_unstale":"2023-07-06T02:22:58.463853+0000","last_undegraded":"2023-07-06T02:22:58.463853+0000","last_fullsized":"2023-07-06T02:22:58.463853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:26.074085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642190+0000","last_change":"2023-07-06T02:22:52.420802+0000","last_active":"2023-07-06T02:22:59.642190+0000","last_peered":"2023-07-06T02:22:59.642190+0000","last_clean":"2023-07-06T02:22:59.642190+0000","last_became_active":"2023-07-06T02:22:52.420494+0000","last_became_peered":"2023-07-06T02:22:52.420494+0000","last_unstale":"2023-07-06T02:22:59.642190+0000","last_undegraded":"2023-07-06T02:22:59.642190+0000","last_fullsized":"2023-07-06T02:22:59.642190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:04.500268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643228+0000","last_change":"2023-07-06T02:22:53.432233+0000","last_active":"2023-07-06T02:22:59.643228+0000","last_peered":"2023-07-06T02:22:59.643228+0000","last_clean":"2023-07-06T02:22:59.643228+0000","last_became_active":"2023-07-06T02:22:53.431893+0000","last_became_peered":"2023-07-06T02:22:53.431893+0000","last_unstale":"2023-07-06T02:22:59.643228+0000","last_undegraded":"2023-07-06T02:22:59.643228+0000","last_fullsized":"2023-07-06T02:22:59.643228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:11.593561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463256+0000","last_change":"2023-07-06T02:22:57.455213+0000","last_active":"2023-07-06T02:22:58.463256+0000","last_peered":"2023-07-06T02:22:58.463256+0000","last_clean":"2023-07-06T02:22:58.463256+0000","last_became_active":"2023-07-06T02:22:57.454800+0000","last_became_peered":"2023-07-06T02:22:57.454800+0000","last_unstale":"2023-07-06T02:22:58.463256+0000","last_undegraded":"2023-07-06T02:22:58.463256+0000","last_fullsized":"2023-07-06T02:22:58.463256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:00.449720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545483+0000","last_change":"2023-07-06T02:22:52.421475+0000","last_active":"2023-07-06T02:22:58.545483+0000","last_peered":"2023-07-06T02:22:58.545483+0000","last_clean":"2023-07-06T02:22:58.545483+0000","last_became_active":"2023-07-06T02:22:52.421267+0000","last_became_peered":"2023-07-06T02:22:52.421267+0000","last_unstale":"2023-07-06T02:22:58.545483+0000","last_undegraded":"2023-07-06T02:22:58.545483+0000","last_fullsized":"2023-07-06T02:22:58.545483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:09.996912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642587+0000","last_change":"2023-07-06T02:22:53.420060+0000","last_active":"2023-07-06T02:22:59.642587+0000","last_peered":"2023-07-06T02:22:59.642587+0000","last_clean":"2023-07-06T02:22:59.642587+0000","last_became_active":"2023-07-06T02:22:53.419347+0000","last_became_peered":"2023-07-06T02:22:53.419347+0000","last_unstale":"2023-07-06T02:22:59.642587+0000","last_undegraded":"2023-07-06T02:22:59.642587+0000","last_fullsized":"2023-07-06T02:22:59.642587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:28.862804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643160+0000","last_change":"2023-07-06T02:22:57.453231+0000","last_active":"2023-07-06T02:22:59.643160+0000","last_peered":"2023-07-06T02:22:59.643160+0000","last_clean":"2023-07-06T02:22:59.643160+0000","last_became_active":"2023-07-06T02:22:57.452776+0000","last_became_peered":"2023-07-06T02:22:57.452776+0000","last_unstale":"2023-07-06T02:22:59.643160+0000","last_undegraded":"2023-07-06T02:22:59.643160+0000","last_fullsized":"2023-07-06T02:22:59.643160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:09.535749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462794+0000","last_change":"2023-07-06T02:22:52.417281+0000","last_active":"2023-07-06T02:22:58.462794+0000","last_peered":"2023-07-06T02:22:58.462794+0000","last_clean":"2023-07-06T02:22:58.462794+0000","last_became_active":"2023-07-06T02:22:52.417011+0000","last_became_peered":"2023-07-06T02:22:52.417011+0000","last_unstale":"2023-07-06T02:22:58.462794+0000","last_undegraded":"2023-07-06T02:22:58.462794+0000","last_fullsized":"2023-07-06T02:22:58.462794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:56.885245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463676+0000","last_change":"2023-07-06T02:22:53.433973+0000","last_active":"2023-07-06T02:22:58.463676+0000","last_peered":"2023-07-06T02:22:58.463676+0000","last_clean":"2023-07-06T02:22:58.463676+0000","last_became_active":"2023-07-06T02:22:53.433561+0000","last_became_peered":"2023-07-06T02:22:53.433561+0000","last_unstale":"2023-07-06T02:22:58.463676+0000","last_undegraded":"2023-07-06T02:22:58.463676+0000","last_fullsized":"2023-07-06T02:22:58.463676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:36.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545710+0000","last_change":"2023-07-06T02:22:57.455016+0000","last_active":"2023-07-06T02:22:58.545710+0000","last_peered":"2023-07-06T02:22:58.545710+0000","last_clean":"2023-07-06T02:22:58.545710+0000","last_became_active":"2023-07-06T02:22:57.454627+0000","last_became_peered":"2023-07-06T02:22:57.454627+0000","last_unstale":"2023-07-06T02:22:58.545710+0000","last_undegraded":"2023-07-06T02:22:58.545710+0000","last_fullsized":"2023-07-06T02:22:58.545710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:44.265890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546232+0000","last_change":"2023-07-06T02:22:52.420410+0000","last_active":"2023-07-06T02:22:58.546232+0000","last_peered":"2023-07-06T02:22:58.546232+0000","last_clean":"2023-07-06T02:22:58.546232+0000","last_became_active":"2023-07-06T02:22:52.420089+0000","last_became_peered":"2023-07-06T02:22:52.420089+0000","last_unstale":"2023-07-06T02:22:58.546232+0000","last_undegraded":"2023-07-06T02:22:58.546232+0000","last_fullsized":"2023-07-06T02:22:58.546232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642490+0000","last_change":"2023-07-06T02:22:53.432339+0000","last_active":"2023-07-06T02:22:59.642490+0000","last_peered":"2023-07-06T02:22:59.642490+0000","last_clean":"2023-07-06T02:22:59.642490+0000","last_became_active":"2023-07-06T02:22:53.431993+0000","last_became_peered":"2023-07-06T02:22:53.431993+0000","last_unstale":"2023-07-06T02:22:59.642490+0000","last_undegraded":"2023-07-06T02:22:59.642490+0000","last_fullsized":"2023-07-06T02:22:59.642490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462370+0000","last_change":"2023-07-06T02:22:57.457792+0000","last_active":"2023-07-06T02:22:58.462370+0000","last_peered":"2023-07-06T02:22:58.462370+0000","last_clean":"2023-07-06T02:22:58.462370+0000","last_became_active":"2023-07-06T02:22:57.457396+0000","last_became_peered":"2023-07-06T02:22:57.457396+0000","last_unstale":"2023-07-06T02:22:58.462370+0000","last_undegraded":"2023-07-06T02:22:58.462370+0000","last_fullsized":"2023-07-06T02:22:58.462370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:17.206072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.546155+0000","last_change":"2023-07-06T02:22:52.421892+0000","last_active":"2023-07-06T02:22:58.546155+0000","last_peered":"2023-07-06T02:22:58.546155+0000","last_clean":"2023-07-06T02:22:58.546155+0000","last_became_active":"2023-07-06T02:22:52.421613+0000","last_became_peered":"2023-07-06T02:22:52.421613+0000","last_unstale":"2023-07-06T02:22:58.546155+0000","last_undegraded":"2023-07-06T02:22:58.546155+0000","last_fullsized":"2023-07-06T02:22:58.546155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622924+0000","last_change":"2023-07-06T02:22:53.422219+0000","last_active":"2023-07-06T02:22:58.622924+0000","last_peered":"2023-07-06T02:22:58.622924+0000","last_clean":"2023-07-06T02:22:58.622924+0000","last_became_active":"2023-07-06T02:22:53.421934+0000","last_became_peered":"2023-07-06T02:22:53.421934+0000","last_unstale":"2023-07-06T02:22:58.622924+0000","last_undegraded":"2023-07-06T02:22:58.622924+0000","last_fullsized":"2023-07-06T02:22:58.622924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:14:41.243585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462824+0000","last_change":"2023-07-06T02:22:57.450611+0000","last_active":"2023-07-06T02:22:58.462824+0000","last_peered":"2023-07-06T02:22:58.462824+0000","last_clean":"2023-07-06T02:22:58.462824+0000","last_became_active":"2023-07-06T02:22:57.450255+0000","last_became_peered":"2023-07-06T02:22:57.450255+0000","last_unstale":"2023-07-06T02:22:58.462824+0000","last_undegraded":"2023-07-06T02:22:58.462824+0000","last_fullsized":"2023-07-06T02:22:58.462824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:04.461198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545892+0000","last_change":"2023-07-06T02:22:52.420450+0000","last_active":"2023-07-06T02:22:58.545892+0000","last_peered":"2023-07-06T02:22:58.545892+0000","last_clean":"2023-07-06T02:22:58.545892+0000","last_became_active":"2023-07-06T02:22:52.420171+0000","last_became_peered":"2023-07-06T02:22:52.420171+0000","last_unstale":"2023-07-06T02:22:58.545892+0000","last_undegraded":"2023-07-06T02:22:58.545892+0000","last_fullsized":"2023-07-06T02:22:58.545892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:35.849912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462735+0000","last_change":"2023-07-06T02:22:53.425285+0000","last_active":"2023-07-06T02:22:58.462735+0000","last_peered":"2023-07-06T02:22:58.462735+0000","last_clean":"2023-07-06T02:22:58.462735+0000","last_became_active":"2023-07-06T02:22:53.424994+0000","last_became_peered":"2023-07-06T02:22:53.424994+0000","last_unstale":"2023-07-06T02:22:58.462735+0000","last_undegraded":"2023-07-06T02:22:58.462735+0000","last_fullsized":"2023-07-06T02:22:58.462735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.580209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462875+0000","last_change":"2023-07-06T02:22:57.459471+0000","last_active":"2023-07-06T02:22:58.462875+0000","last_peered":"2023-07-06T02:22:58.462875+0000","last_clean":"2023-07-06T02:22:58.462875+0000","last_became_active":"2023-07-06T02:22:57.459086+0000","last_became_peered":"2023-07-06T02:22:57.459086+0000","last_unstale":"2023-07-06T02:22:58.462875+0000","last_undegraded":"2023-07-06T02:22:58.462875+0000","last_fullsized":"2023-07-06T02:22:58.462875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:32.995367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643074+0000","last_change":"2023-07-06T02:22:52.419379+0000","last_active":"2023-07-06T02:22:59.643074+0000","last_peered":"2023-07-06T02:22:59.643074+0000","last_clean":"2023-07-06T02:22:59.643074+0000","last_became_active":"2023-07-06T02:22:52.419100+0000","last_became_peered":"2023-07-06T02:22:52.419100+0000","last_unstale":"2023-07-06T02:22:59.643074+0000","last_undegraded":"2023-07-06T02:22:59.643074+0000","last_fullsized":"2023-07-06T02:22:59.643074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:15.612393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642483+0000","last_change":"2023-07-06T02:22:53.423846+0000","last_active":"2023-07-06T02:22:59.642483+0000","last_peered":"2023-07-06T02:22:59.642483+0000","last_clean":"2023-07-06T02:22:59.642483+0000","last_became_active":"2023-07-06T02:22:53.423585+0000","last_became_peered":"2023-07-06T02:22:53.423585+0000","last_unstale":"2023-07-06T02:22:59.642483+0000","last_undegraded":"2023-07-06T02:22:59.642483+0000","last_fullsized":"2023-07-06T02:22:59.642483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:05.040887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462786+0000","last_change":"2023-07-06T02:22:57.450230+0000","last_active":"2023-07-06T02:22:58.462786+0000","last_peered":"2023-07-06T02:22:58.462786+0000","last_clean":"2023-07-06T02:22:58.462786+0000","last_became_active":"2023-07-06T02:22:57.449841+0000","last_became_peered":"2023-07-06T02:22:57.449841+0000","last_unstale":"2023-07-06T02:22:58.462786+0000","last_undegraded":"2023-07-06T02:22:58.462786+0000","last_fullsized":"2023-07-06T02:22:58.462786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:26:07.959918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462612+0000","last_change":"2023-07-06T02:22:52.418001+0000","last_active":"2023-07-06T02:22:58.462612+0000","last_peered":"2023-07-06T02:22:58.462612+0000","last_clean":"2023-07-06T02:22:58.462612+0000","last_became_active":"2023-07-06T02:22:52.417737+0000","last_became_peered":"2023-07-06T02:22:52.417737+0000","last_unstale":"2023-07-06T02:22:58.462612+0000","last_undegraded":"2023-07-06T02:22:58.462612+0000","last_fullsized":"2023-07-06T02:22:58.462612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:32.987370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623068+0000","last_change":"2023-07-06T02:22:53.418072+0000","last_active":"2023-07-06T02:22:58.623068+0000","last_peered":"2023-07-06T02:22:58.623068+0000","last_clean":"2023-07-06T02:22:58.623068+0000","last_became_active":"2023-07-06T02:22:53.416070+0000","last_became_peered":"2023-07-06T02:22:53.416070+0000","last_unstale":"2023-07-06T02:22:58.623068+0000","last_undegraded":"2023-07-06T02:22:58.623068+0000","last_fullsized":"2023-07-06T02:22:58.623068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:39:55.241653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497345+0000","last_change":"2023-07-06T02:22:57.451350+0000","last_active":"2023-07-06T02:22:58.497345+0000","last_peered":"2023-07-06T02:22:58.497345+0000","last_clean":"2023-07-06T02:22:58.497345+0000","last_became_active":"2023-07-06T02:22:57.450988+0000","last_became_peered":"2023-07-06T02:22:57.450988+0000","last_unstale":"2023-07-06T02:22:58.497345+0000","last_undegraded":"2023-07-06T02:22:58.497345+0000","last_fullsized":"2023-07-06T02:22:58.497345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:08:02.280780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462907+0000","last_change":"2023-07-06T02:22:52.423191+0000","last_active":"2023-07-06T02:22:58.462907+0000","last_peered":"2023-07-06T02:22:58.462907+0000","last_clean":"2023-07-06T02:22:58.462907+0000","last_became_active":"2023-07-06T02:22:52.422863+0000","last_became_peered":"2023-07-06T02:22:52.422863+0000","last_unstale":"2023-07-06T02:22:58.462907+0000","last_undegraded":"2023-07-06T02:22:58.462907+0000","last_fullsized":"2023-07-06T02:22:58.462907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:22.727110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623436+0000","last_change":"2023-07-06T02:22:53.419350+0000","last_active":"2023-07-06T02:22:58.623436+0000","last_peered":"2023-07-06T02:22:58.623436+0000","last_clean":"2023-07-06T02:22:58.623436+0000","last_became_active":"2023-07-06T02:22:53.419024+0000","last_became_peered":"2023-07-06T02:22:53.419024+0000","last_unstale":"2023-07-06T02:22:58.623436+0000","last_undegraded":"2023-07-06T02:22:58.623436+0000","last_fullsized":"2023-07-06T02:22:58.623436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:33.746001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643344+0000","last_change":"2023-07-06T02:22:57.455212+0000","last_active":"2023-07-06T02:22:59.643344+0000","last_peered":"2023-07-06T02:22:59.643344+0000","last_clean":"2023-07-06T02:22:59.643344+0000","last_became_active":"2023-07-06T02:22:57.454773+0000","last_became_peered":"2023-07-06T02:22:57.454773+0000","last_unstale":"2023-07-06T02:22:59.643344+0000","last_undegraded":"2023-07-06T02:22:59.643344+0000","last_fullsized":"2023-07-06T02:22:59.643344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622698+0000","last_change":"2023-07-06T02:22:52.417326+0000","last_active":"2023-07-06T02:22:58.622698+0000","last_peered":"2023-07-06T02:22:58.622698+0000","last_clean":"2023-07-06T02:22:58.622698+0000","last_became_active":"2023-07-06T02:22:52.417065+0000","last_became_peered":"2023-07-06T02:22:52.417065+0000","last_unstale":"2023-07-06T02:22:58.622698+0000","last_undegraded":"2023-07-06T02:22:58.622698+0000","last_fullsized":"2023-07-06T02:22:58.622698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:35.235033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623747+0000","last_change":"2023-07-06T02:22:53.422511+0000","last_active":"2023-07-06T02:22:58.623747+0000","last_peered":"2023-07-06T02:22:58.623747+0000","last_clean":"2023-07-06T02:22:58.623747+0000","last_became_active":"2023-07-06T02:22:53.420266+0000","last_became_peered":"2023-07-06T02:22:53.420266+0000","last_unstale":"2023-07-06T02:22:58.623747+0000","last_undegraded":"2023-07-06T02:22:58.623747+0000","last_fullsized":"2023-07-06T02:22:58.623747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:05.329078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623647+0000","last_change":"2023-07-06T02:22:57.451156+0000","last_active":"2023-07-06T02:22:58.623647+0000","last_peered":"2023-07-06T02:22:58.623647+0000","last_clean":"2023-07-06T02:22:58.623647+0000","last_became_active":"2023-07-06T02:22:57.450784+0000","last_became_peered":"2023-07-06T02:22:57.450784+0000","last_unstale":"2023-07-06T02:22:58.623647+0000","last_undegraded":"2023-07-06T02:22:58.623647+0000","last_fullsized":"2023-07-06T02:22:58.623647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:15:13.002024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462668+0000","last_change":"2023-07-06T02:22:52.418727+0000","last_active":"2023-07-06T02:22:58.462668+0000","last_peered":"2023-07-06T02:22:58.462668+0000","last_clean":"2023-07-06T02:22:58.462668+0000","last_became_active":"2023-07-06T02:22:52.418454+0000","last_became_peered":"2023-07-06T02:22:52.418454+0000","last_unstale":"2023-07-06T02:22:58.462668+0000","last_undegraded":"2023-07-06T02:22:58.462668+0000","last_fullsized":"2023-07-06T02:22:58.462668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:05.425463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497348+0000","last_change":"2023-07-06T02:22:53.432617+0000","last_active":"2023-07-06T02:22:58.497348+0000","last_peered":"2023-07-06T02:22:58.497348+0000","last_clean":"2023-07-06T02:22:58.497348+0000","last_became_active":"2023-07-06T02:22:53.432332+0000","last_became_peered":"2023-07-06T02:22:53.432332+0000","last_unstale":"2023-07-06T02:22:58.497348+0000","last_undegraded":"2023-07-06T02:22:58.497348+0000","last_fullsized":"2023-07-06T02:22:58.497348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:16.619713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623091+0000","last_change":"2023-07-06T02:22:57.463354+0000","last_active":"2023-07-06T02:22:58.623091+0000","last_peered":"2023-07-06T02:22:58.623091+0000","last_clean":"2023-07-06T02:22:58.623091+0000","last_became_active":"2023-07-06T02:22:57.462934+0000","last_became_peered":"2023-07-06T02:22:57.462934+0000","last_unstale":"2023-07-06T02:22:58.623091+0000","last_undegraded":"2023-07-06T02:22:58.623091+0000","last_fullsized":"2023-07-06T02:22:58.623091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:23.690834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463354+0000","last_change":"2023-07-06T02:22:52.421219+0000","last_active":"2023-07-06T02:22:58.463354+0000","last_peered":"2023-07-06T02:22:58.463354+0000","last_clean":"2023-07-06T02:22:58.463354+0000","last_became_active":"2023-07-06T02:22:52.420927+0000","last_became_peered":"2023-07-06T02:22:52.420927+0000","last_unstale":"2023-07-06T02:22:58.463354+0000","last_undegraded":"2023-07-06T02:22:58.463354+0000","last_fullsized":"2023-07-06T02:22:58.463354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:17:11.011220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623582+0000","last_change":"2023-07-06T02:22:47.973696+0000","last_active":"2023-07-06T02:22:58.623582+0000","last_peered":"2023-07-06T02:22:58.623582+0000","last_clean":"2023-07-06T02:22:58.623582+0000","last_became_active":"2023-07-06T02:22:47.973358+0000","last_became_peered":"2023-07-06T02:22:47.973358+0000","last_unstale":"2023-07-06T02:22:58.623582+0000","last_undegraded":"2023-07-06T02:22:58.623582+0000","last_fullsized":"2023-07-06T02:22:58.623582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:26:55.060583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462551+0000","last_change":"2023-07-06T02:22:53.433182+0000","last_active":"2023-07-06T02:22:58.462551+0000","last_peered":"2023-07-06T02:22:58.462551+0000","last_clean":"2023-07-06T02:22:58.462551+0000","last_became_active":"2023-07-06T02:22:53.432900+0000","last_became_peered":"2023-07-06T02:22:53.432900+0000","last_unstale":"2023-07-06T02:22:58.462551+0000","last_undegraded":"2023-07-06T02:22:58.462551+0000","last_fullsized":"2023-07-06T02:22:58.462551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462417+0000","last_change":"2023-07-06T02:22:57.456985+0000","last_active":"2023-07-06T02:22:58.462417+0000","last_peered":"2023-07-06T02:22:58.462417+0000","last_clean":"2023-07-06T02:22:58.462417+0000","last_became_active":"2023-07-06T02:22:57.456594+0000","last_became_peered":"2023-07-06T02:22:57.456594+0000","last_unstale":"2023-07-06T02:22:58.462417+0000","last_undegraded":"2023-07-06T02:22:58.462417+0000","last_fullsized":"2023-07-06T02:22:58.462417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:44.241938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463270+0000","last_change":"2023-07-06T02:22:47.323237+0000","last_active":"2023-07-06T02:22:58.463270+0000","last_peered":"2023-07-06T02:22:58.463270+0000","last_clean":"2023-07-06T02:22:58.463270+0000","last_became_active":"2023-07-06T02:22:47.322763+0000","last_became_peered":"2023-07-06T02:22:47.322763+0000","last_unstale":"2023-07-06T02:22:58.463270+0000","last_undegraded":"2023-07-06T02:22:58.463270+0000","last_fullsized":"2023-07-06T02:22:58.463270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:51:10.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642236+0000","last_change":"2023-07-06T02:22:52.417097+0000","last_active":"2023-07-06T02:22:59.642236+0000","last_peered":"2023-07-06T02:22:59.642236+0000","last_clean":"2023-07-06T02:22:59.642236+0000","last_became_active":"2023-07-06T02:22:52.416824+0000","last_became_peered":"2023-07-06T02:22:52.416824+0000","last_unstale":"2023-07-06T02:22:59.642236+0000","last_undegraded":"2023-07-06T02:22:59.642236+0000","last_fullsized":"2023-07-06T02:22:59.642236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:35.415046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622712+0000","last_change":"2023-07-06T02:22:53.419559+0000","last_active":"2023-07-06T02:22:58.622712+0000","last_peered":"2023-07-06T02:22:58.622712+0000","last_clean":"2023-07-06T02:22:58.622712+0000","last_became_active":"2023-07-06T02:22:53.419069+0000","last_became_peered":"2023-07-06T02:22:53.419069+0000","last_unstale":"2023-07-06T02:22:58.622712+0000","last_undegraded":"2023-07-06T02:22:58.622712+0000","last_fullsized":"2023-07-06T02:22:58.622712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:22.045924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643230+0000","last_change":"2023-07-06T02:22:57.459369+0000","last_active":"2023-07-06T02:22:59.643230+0000","last_peered":"2023-07-06T02:22:59.643230+0000","last_clean":"2023-07-06T02:22:59.643230+0000","last_became_active":"2023-07-06T02:22:57.458965+0000","last_became_peered":"2023-07-06T02:22:57.458965+0000","last_unstale":"2023-07-06T02:22:59.643230+0000","last_undegraded":"2023-07-06T02:22:59.643230+0000","last_fullsized":"2023-07-06T02:22:59.643230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:40.450527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496831+0000","last_change":"2023-07-06T02:22:47.325598+0000","last_active":"2023-07-06T02:22:58.496831+0000","last_peered":"2023-07-06T02:22:58.496831+0000","last_clean":"2023-07-06T02:22:58.496831+0000","last_became_active":"2023-07-06T02:22:47.325315+0000","last_became_peered":"2023-07-06T02:22:47.325315+0000","last_unstale":"2023-07-06T02:22:58.496831+0000","last_undegraded":"2023-07-06T02:22:58.496831+0000","last_fullsized":"2023-07-06T02:22:58.496831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:36.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545612+0000","last_change":"2023-07-06T02:22:47.326882+0000","last_active":"2023-07-06T02:22:58.545612+0000","last_peered":"2023-07-06T02:22:58.545612+0000","last_clean":"2023-07-06T02:22:58.545612+0000","last_became_active":"2023-07-06T02:22:47.326511+0000","last_became_peered":"2023-07-06T02:22:47.326511+0000","last_unstale":"2023-07-06T02:22:58.545612+0000","last_undegraded":"2023-07-06T02:22:58.545612+0000","last_fullsized":"2023-07-06T02:22:58.545612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:46.985907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.641990+0000","last_change":"2023-07-06T02:22:52.416206+0000","last_active":"2023-07-06T02:22:59.641990+0000","last_peered":"2023-07-06T02:22:59.641990+0000","last_clean":"2023-07-06T02:22:59.641990+0000","last_became_active":"2023-07-06T02:22:52.415905+0000","last_became_peered":"2023-07-06T02:22:52.415905+0000","last_unstale":"2023-07-06T02:22:59.641990+0000","last_undegraded":"2023-07-06T02:22:59.641990+0000","last_fullsized":"2023-07-06T02:22:59.641990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:59.401932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642535+0000","last_change":"2023-07-06T02:22:53.423988+0000","last_active":"2023-07-06T02:22:59.642535+0000","last_peered":"2023-07-06T02:22:59.642535+0000","last_clean":"2023-07-06T02:22:59.642535+0000","last_became_active":"2023-07-06T02:22:53.423711+0000","last_became_peered":"2023-07-06T02:22:53.423711+0000","last_unstale":"2023-07-06T02:22:59.642535+0000","last_undegraded":"2023-07-06T02:22:59.642535+0000","last_fullsized":"2023-07-06T02:22:59.642535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463130+0000","last_change":"2023-07-06T02:22:57.453513+0000","last_active":"2023-07-06T02:22:58.463130+0000","last_peered":"2023-07-06T02:22:58.463130+0000","last_clean":"2023-07-06T02:22:58.463130+0000","last_became_active":"2023-07-06T02:22:57.453163+0000","last_became_peered":"2023-07-06T02:22:57.453163+0000","last_unstale":"2023-07-06T02:22:58.463130+0000","last_undegraded":"2023-07-06T02:22:58.463130+0000","last_fullsized":"2023-07-06T02:22:58.463130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:06:26.324856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545616+0000","last_change":"2023-07-06T02:22:47.613219+0000","last_active":"2023-07-06T02:22:58.545616+0000","last_peered":"2023-07-06T02:22:58.545616+0000","last_clean":"2023-07-06T02:22:58.545616+0000","last_became_active":"2023-07-06T02:22:47.612787+0000","last_became_peered":"2023-07-06T02:22:47.612787+0000","last_unstale":"2023-07-06T02:22:58.545616+0000","last_undegraded":"2023-07-06T02:22:58.545616+0000","last_fullsized":"2023-07-06T02:22:58.545616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:24.728713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496749+0000","last_change":"2023-07-06T02:22:52.421146+0000","last_active":"2023-07-06T02:22:58.496749+0000","last_peered":"2023-07-06T02:22:58.496749+0000","last_clean":"2023-07-06T02:22:58.496749+0000","last_became_active":"2023-07-06T02:22:52.420937+0000","last_became_peered":"2023-07-06T02:22:52.420937+0000","last_unstale":"2023-07-06T02:22:58.496749+0000","last_undegraded":"2023-07-06T02:22:58.496749+0000","last_fullsized":"2023-07-06T02:22:58.496749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:41.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497242+0000","last_change":"2023-07-06T02:22:53.428966+0000","last_active":"2023-07-06T02:22:58.497242+0000","last_peered":"2023-07-06T02:22:58.497242+0000","last_clean":"2023-07-06T02:22:58.497242+0000","last_became_active":"2023-07-06T02:22:53.428644+0000","last_became_peered":"2023-07-06T02:22:53.428644+0000","last_unstale":"2023-07-06T02:22:58.497242+0000","last_undegraded":"2023-07-06T02:22:58.497242+0000","last_fullsized":"2023-07-06T02:22:58.497242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:10.006953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545738+0000","last_change":"2023-07-06T02:22:57.445443+0000","last_active":"2023-07-06T02:22:58.545738+0000","last_peered":"2023-07-06T02:22:58.545738+0000","last_clean":"2023-07-06T02:22:58.545738+0000","last_became_active":"2023-07-06T02:22:57.445042+0000","last_became_peered":"2023-07-06T02:22:57.445042+0000","last_unstale":"2023-07-06T02:22:58.545738+0000","last_undegraded":"2023-07-06T02:22:58.545738+0000","last_fullsized":"2023-07-06T02:22:58.545738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:09.069048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545450+0000","last_change":"2023-07-06T02:22:47.613102+0000","last_active":"2023-07-06T02:22:58.545450+0000","last_peered":"2023-07-06T02:22:58.545450+0000","last_clean":"2023-07-06T02:22:58.545450+0000","last_became_active":"2023-07-06T02:22:47.612655+0000","last_became_peered":"2023-07-06T02:22:47.612655+0000","last_unstale":"2023-07-06T02:22:58.545450+0000","last_undegraded":"2023-07-06T02:22:58.545450+0000","last_fullsized":"2023-07-06T02:22:58.545450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:30.336091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462609+0000","last_change":"2023-07-06T02:22:52.419398+0000","last_active":"2023-07-06T02:22:58.462609+0000","last_peered":"2023-07-06T02:22:58.462609+0000","last_clean":"2023-07-06T02:22:58.462609+0000","last_became_active":"2023-07-06T02:22:52.419139+0000","last_became_peered":"2023-07-06T02:22:52.419139+0000","last_unstale":"2023-07-06T02:22:58.462609+0000","last_undegraded":"2023-07-06T02:22:58.462609+0000","last_fullsized":"2023-07-06T02:22:58.462609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463313+0000","last_change":"2023-07-06T02:22:53.425795+0000","last_active":"2023-07-06T02:22:58.463313+0000","last_peered":"2023-07-06T02:22:58.463313+0000","last_clean":"2023-07-06T02:22:58.463313+0000","last_became_active":"2023-07-06T02:22:53.425518+0000","last_became_peered":"2023-07-06T02:22:53.425518+0000","last_unstale":"2023-07-06T02:22:58.463313+0000","last_undegraded":"2023-07-06T02:22:58.463313+0000","last_fullsized":"2023-07-06T02:22:58.463313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:21:40.141734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642407+0000","last_change":"2023-07-06T02:22:57.457031+0000","last_active":"2023-07-06T02:22:59.642407+0000","last_peered":"2023-07-06T02:22:59.642407+0000","last_clean":"2023-07-06T02:22:59.642407+0000","last_became_active":"2023-07-06T02:22:57.456653+0000","last_became_peered":"2023-07-06T02:22:57.456653+0000","last_unstale":"2023-07-06T02:22:59.642407+0000","last_undegraded":"2023-07-06T02:22:59.642407+0000","last_fullsized":"2023-07-06T02:22:59.642407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:34.843149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642558+0000","last_change":"2023-07-06T02:22:47.613009+0000","last_active":"2023-07-06T02:22:59.642558+0000","last_peered":"2023-07-06T02:22:59.642558+0000","last_clean":"2023-07-06T02:22:59.642558+0000","last_became_active":"2023-07-06T02:22:47.612664+0000","last_became_peered":"2023-07-06T02:22:47.612664+0000","last_unstale":"2023-07-06T02:22:59.642558+0000","last_undegraded":"2023-07-06T02:22:59.642558+0000","last_fullsized":"2023-07-06T02:22:59.642558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:52:35.690988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623032+0000","last_change":"2023-07-06T02:22:52.417974+0000","last_active":"2023-07-06T02:22:58.623032+0000","last_peered":"2023-07-06T02:22:58.623032+0000","last_clean":"2023-07-06T02:22:58.623032+0000","last_became_active":"2023-07-06T02:22:52.417681+0000","last_became_peered":"2023-07-06T02:22:52.417681+0000","last_unstale":"2023-07-06T02:22:58.623032+0000","last_undegraded":"2023-07-06T02:22:58.623032+0000","last_fullsized":"2023-07-06T02:22:58.623032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:28:34.626494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496776+0000","last_change":"2023-07-06T02:22:53.424117+0000","last_active":"2023-07-06T02:22:58.496776+0000","last_peered":"2023-07-06T02:22:58.496776+0000","last_clean":"2023-07-06T02:22:58.496776+0000","last_became_active":"2023-07-06T02:22:53.423755+0000","last_became_peered":"2023-07-06T02:22:53.423755+0000","last_unstale":"2023-07-06T02:22:58.496776+0000","last_undegraded":"2023-07-06T02:22:58.496776+0000","last_fullsized":"2023-07-06T02:22:58.496776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:48.677462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463117+0000","last_change":"2023-07-06T02:22:57.451130+0000","last_active":"2023-07-06T02:22:58.463117+0000","last_peered":"2023-07-06T02:22:58.463117+0000","last_clean":"2023-07-06T02:22:58.463117+0000","last_became_active":"2023-07-06T02:22:57.450726+0000","last_became_peered":"2023-07-06T02:22:57.450726+0000","last_unstale":"2023-07-06T02:22:58.463117+0000","last_undegraded":"2023-07-06T02:22:58.463117+0000","last_fullsized":"2023-07-06T02:22:58.463117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:38.492011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463258+0000","last_change":"2023-07-06T02:22:47.612949+0000","last_active":"2023-07-06T02:22:58.463258+0000","last_peered":"2023-07-06T02:22:58.463258+0000","last_clean":"2023-07-06T02:22:58.463258+0000","last_became_active":"2023-07-06T02:22:47.612606+0000","last_became_peered":"2023-07-06T02:22:47.612606+0000","last_unstale":"2023-07-06T02:22:58.463258+0000","last_undegraded":"2023-07-06T02:22:58.463258+0000","last_fullsized":"2023-07-06T02:22:58.463258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:26:58.278350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496991+0000","last_change":"2023-07-06T02:22:52.421787+0000","last_active":"2023-07-06T02:22:58.496991+0000","last_peered":"2023-07-06T02:22:58.496991+0000","last_clean":"2023-07-06T02:22:58.496991+0000","last_became_active":"2023-07-06T02:22:52.421536+0000","last_became_peered":"2023-07-06T02:22:52.421536+0000","last_unstale":"2023-07-06T02:22:58.496991+0000","last_undegraded":"2023-07-06T02:22:58.496991+0000","last_fullsized":"2023-07-06T02:22:58.496991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:50.380161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642624+0000","last_change":"2023-07-06T02:22:53.419643+0000","last_active":"2023-07-06T02:22:59.642624+0000","last_peered":"2023-07-06T02:22:59.642624+0000","last_clean":"2023-07-06T02:22:59.642624+0000","last_became_active":"2023-07-06T02:22:53.418153+0000","last_became_peered":"2023-07-06T02:22:53.418153+0000","last_unstale":"2023-07-06T02:22:59.642624+0000","last_undegraded":"2023-07-06T02:22:59.642624+0000","last_fullsized":"2023-07-06T02:22:59.642624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:30:15.982160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642319+0000","last_change":"2023-07-06T02:22:57.455969+0000","last_active":"2023-07-06T02:22:59.642319+0000","last_peered":"2023-07-06T02:22:59.642319+0000","last_clean":"2023-07-06T02:22:59.642319+0000","last_became_active":"2023-07-06T02:22:57.455566+0000","last_became_peered":"2023-07-06T02:22:57.455566+0000","last_unstale":"2023-07-06T02:22:59.642319+0000","last_undegraded":"2023-07-06T02:22:59.642319+0000","last_fullsized":"2023-07-06T02:22:59.642319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:15.032393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463409+0000","last_change":"2023-07-06T02:22:47.329340+0000","last_active":"2023-07-06T02:22:58.463409+0000","last_peered":"2023-07-06T02:22:58.463409+0000","last_clean":"2023-07-06T02:22:58.463409+0000","last_became_active":"2023-07-06T02:22:47.329090+0000","last_became_peered":"2023-07-06T02:22:47.329090+0000","last_unstale":"2023-07-06T02:22:58.463409+0000","last_undegraded":"2023-07-06T02:22:58.463409+0000","last_fullsized":"2023-07-06T02:22:58.463409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:42.929148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643052+0000","last_change":"2023-07-06T02:22:52.422406+0000","last_active":"2023-07-06T02:22:59.643052+0000","last_peered":"2023-07-06T02:22:59.643052+0000","last_clean":"2023-07-06T02:22:59.643052+0000","last_became_active":"2023-07-06T02:22:52.422125+0000","last_became_peered":"2023-07-06T02:22:52.422125+0000","last_unstale":"2023-07-06T02:22:59.643052+0000","last_undegraded":"2023-07-06T02:22:59.643052+0000","last_fullsized":"2023-07-06T02:22:59.643052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:08.999634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496575+0000","last_change":"2023-07-06T02:22:53.428897+0000","last_active":"2023-07-06T02:22:58.496575+0000","last_peered":"2023-07-06T02:22:58.496575+0000","last_clean":"2023-07-06T02:22:58.496575+0000","last_became_active":"2023-07-06T02:22:53.428651+0000","last_became_peered":"2023-07-06T02:22:53.428651+0000","last_unstale":"2023-07-06T02:22:58.496575+0000","last_undegraded":"2023-07-06T02:22:58.496575+0000","last_fullsized":"2023-07-06T02:22:58.496575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:49.714737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.497099+0000","last_change":"2023-07-06T02:22:57.454674+0000","last_active":"2023-07-06T02:22:58.497099+0000","last_peered":"2023-07-06T02:22:58.497099+0000","last_clean":"2023-07-06T02:22:58.497099+0000","last_became_active":"2023-07-06T02:22:57.454162+0000","last_became_peered":"2023-07-06T02:22:57.454162+0000","last_unstale":"2023-07-06T02:22:58.497099+0000","last_undegraded":"2023-07-06T02:22:58.497099+0000","last_fullsized":"2023-07-06T02:22:58.497099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:36:56.768674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642632+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:59.642632+0000","last_peered":"2023-07-06T02:22:59.642632+0000","last_clean":"2023-07-06T02:22:59.642632+0000","last_became_active":"2023-07-06T02:22:52.418794+0000","last_became_peered":"2023-07-06T02:22:52.418794+0000","last_unstale":"2023-07-06T02:22:59.642632+0000","last_undegraded":"2023-07-06T02:22:59.642632+0000","last_fullsized":"2023-07-06T02:22:59.642632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:33.168769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463505+0000","last_change":"2023-07-06T02:22:53.424846+0000","last_active":"2023-07-06T02:22:58.463505+0000","last_peered":"2023-07-06T02:22:58.463505+0000","last_clean":"2023-07-06T02:22:58.463505+0000","last_became_active":"2023-07-06T02:22:53.424606+0000","last_became_peered":"2023-07-06T02:22:53.424606+0000","last_unstale":"2023-07-06T02:22:58.463505+0000","last_undegraded":"2023-07-06T02:22:58.463505+0000","last_fullsized":"2023-07-06T02:22:58.463505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:31:39.592702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.545637+0000","last_change":"2023-07-06T02:22:57.455280+0000","last_active":"2023-07-06T02:22:58.545637+0000","last_peered":"2023-07-06T02:22:58.545637+0000","last_clean":"2023-07-06T02:22:58.545637+0000","last_became_active":"2023-07-06T02:22:57.454860+0000","last_became_peered":"2023-07-06T02:22:57.454860+0000","last_unstale":"2023-07-06T02:22:58.545637+0000","last_undegraded":"2023-07-06T02:22:58.545637+0000","last_fullsized":"2023-07-06T02:22:58.545637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:11.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642892+0000","last_change":"2023-07-06T02:22:52.419322+0000","last_active":"2023-07-06T02:22:59.642892+0000","last_peered":"2023-07-06T02:22:59.642892+0000","last_clean":"2023-07-06T02:22:59.642892+0000","last_became_active":"2023-07-06T02:22:52.418950+0000","last_became_peered":"2023-07-06T02:22:52.418950+0000","last_unstale":"2023-07-06T02:22:59.642892+0000","last_undegraded":"2023-07-06T02:22:59.642892+0000","last_fullsized":"2023-07-06T02:22:59.642892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:22.417730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623010+0000","last_change":"2023-07-06T02:22:53.419784+0000","last_active":"2023-07-06T02:22:58.623010+0000","last_peered":"2023-07-06T02:22:58.623010+0000","last_clean":"2023-07-06T02:22:58.623010+0000","last_became_active":"2023-07-06T02:22:53.419477+0000","last_became_peered":"2023-07-06T02:22:53.419477+0000","last_unstale":"2023-07-06T02:22:58.623010+0000","last_undegraded":"2023-07-06T02:22:58.623010+0000","last_fullsized":"2023-07-06T02:22:58.623010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:07.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643049+0000","last_change":"2023-07-06T02:22:57.447138+0000","last_active":"2023-07-06T02:22:59.643049+0000","last_peered":"2023-07-06T02:22:59.643049+0000","last_clean":"2023-07-06T02:22:59.643049+0000","last_became_active":"2023-07-06T02:22:57.446786+0000","last_became_peered":"2023-07-06T02:22:57.446786+0000","last_unstale":"2023-07-06T02:22:59.643049+0000","last_undegraded":"2023-07-06T02:22:59.643049+0000","last_fullsized":"2023-07-06T02:22:59.643049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:45.545737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.643069+0000","last_change":"2023-07-06T02:22:52.419733+0000","last_active":"2023-07-06T02:22:59.643069+0000","last_peered":"2023-07-06T02:22:59.643069+0000","last_clean":"2023-07-06T02:22:59.643069+0000","last_became_active":"2023-07-06T02:22:52.418972+0000","last_became_peered":"2023-07-06T02:22:52.418972+0000","last_unstale":"2023-07-06T02:22:59.643069+0000","last_undegraded":"2023-07-06T02:22:59.643069+0000","last_fullsized":"2023-07-06T02:22:59.643069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:07:57.245299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622844+0000","last_change":"2023-07-06T02:22:53.416731+0000","last_active":"2023-07-06T02:22:58.622844+0000","last_peered":"2023-07-06T02:22:58.622844+0000","last_clean":"2023-07-06T02:22:58.622844+0000","last_became_active":"2023-07-06T02:22:53.416412+0000","last_became_peered":"2023-07-06T02:22:53.416412+0000","last_unstale":"2023-07-06T02:22:58.622844+0000","last_undegraded":"2023-07-06T02:22:58.622844+0000","last_fullsized":"2023-07-06T02:22:58.622844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:36:20.268819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622716+0000","last_change":"2023-07-06T02:22:57.461025+0000","last_active":"2023-07-06T02:22:58.622716+0000","last_peered":"2023-07-06T02:22:58.622716+0000","last_clean":"2023-07-06T02:22:58.622716+0000","last_became_active":"2023-07-06T02:22:57.460633+0000","last_became_peered":"2023-07-06T02:22:57.460633+0000","last_unstale":"2023-07-06T02:22:58.622716+0000","last_undegraded":"2023-07-06T02:22:58.622716+0000","last_fullsized":"2023-07-06T02:22:58.622716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:26.873513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463123+0000","last_change":"2023-07-06T02:22:52.419040+0000","last_active":"2023-07-06T02:22:58.463123+0000","last_peered":"2023-07-06T02:22:58.463123+0000","last_clean":"2023-07-06T02:22:58.463123+0000","last_became_active":"2023-07-06T02:22:52.418771+0000","last_became_peered":"2023-07-06T02:22:52.418771+0000","last_unstale":"2023-07-06T02:22:58.463123+0000","last_undegraded":"2023-07-06T02:22:58.463123+0000","last_fullsized":"2023-07-06T02:22:58.463123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:24:21.889565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.463056+0000","last_change":"2023-07-06T02:22:53.421840+0000","last_active":"2023-07-06T02:22:58.463056+0000","last_peered":"2023-07-06T02:22:58.463056+0000","last_clean":"2023-07-06T02:22:58.463056+0000","last_became_active":"2023-07-06T02:22:53.421644+0000","last_became_peered":"2023-07-06T02:22:53.421644+0000","last_unstale":"2023-07-06T02:22:58.463056+0000","last_undegraded":"2023-07-06T02:22:58.463056+0000","last_fullsized":"2023-07-06T02:22:58.463056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:42:56.076946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:57.459681+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:57.459298+0000","last_became_peered":"2023-07-06T02:22:57.459298+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:56.424061+0000","last_clean_scrub_stamp":"2023-07-06T02:22:56.424061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:49.871715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:59.642530+0000","last_change":"2023-07-06T02:22:52.425605+0000","last_active":"2023-07-06T02:22:59.642530+0000","last_peered":"2023-07-06T02:22:59.642530+0000","last_clean":"2023-07-06T02:22:59.642530+0000","last_became_active":"2023-07-06T02:22:52.425345+0000","last_became_peered":"2023-07-06T02:22:52.425345+0000","last_unstale":"2023-07-06T02:22:59.642530+0000","last_undegraded":"2023-07-06T02:22:59.642530+0000","last_fullsized":"2023-07-06T02:22:59.642530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:51.388580+0000","last_clean_scrub_stamp":"2023-07-06T02:22:51.388580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:39.206458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.622815+0000","last_change":"2023-07-06T02:22:53.422473+0000","last_active":"2023-07-06T02:22:58.622815+0000","last_peered":"2023-07-06T02:22:58.622815+0000","last_clean":"2023-07-06T02:22:58.622815+0000","last_became_active":"2023-07-06T02:22:53.422171+0000","last_became_peered":"2023-07-06T02:22:53.422171+0000","last_unstale":"2023-07-06T02:22:58.622815+0000","last_undegraded":"2023-07-06T02:22:58.622815+0000","last_fullsized":"2023-07-06T02:22:58.622815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:35.957039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462966+0000","last_change":"2023-07-06T02:22:53.420559+0000","last_active":"2023-07-06T02:22:58.462966+0000","last_peered":"2023-07-06T02:22:58.462966+0000","last_clean":"2023-07-06T02:22:58.462966+0000","last_became_active":"2023-07-06T02:22:53.420220+0000","last_became_peered":"2023-07-06T02:22:53.420220+0000","last_unstale":"2023-07-06T02:22:58.462966+0000","last_undegraded":"2023-07-06T02:22:58.462966+0000","last_fullsized":"2023-07-06T02:22:58.462966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:11.617347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462676+0000","last_change":"2023-07-06T02:22:53.424835+0000","last_active":"2023-07-06T02:22:58.462676+0000","last_peered":"2023-07-06T02:22:58.462676+0000","last_clean":"2023-07-06T02:22:58.462676+0000","last_became_active":"2023-07-06T02:22:53.424520+0000","last_became_peered":"2023-07-06T02:22:53.424520+0000","last_unstale":"2023-07-06T02:22:58.462676+0000","last_undegraded":"2023-07-06T02:22:58.462676+0000","last_fullsized":"2023-07-06T02:22:58.462676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:27.623742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.623256+0000","last_change":"2023-07-06T02:22:53.420661+0000","last_active":"2023-07-06T02:22:58.623256+0000","last_peered":"2023-07-06T02:22:58.623256+0000","last_clean":"2023-07-06T02:22:58.623256+0000","last_became_active":"2023-07-06T02:22:53.420137+0000","last_became_peered":"2023-07-06T02:22:53.420137+0000","last_unstale":"2023-07-06T02:22:58.623256+0000","last_undegraded":"2023-07-06T02:22:58.623256+0000","last_fullsized":"2023-07-06T02:22:58.623256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:07:51.965833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.462455+0000","last_change":"2023-07-06T02:22:53.434477+0000","last_active":"2023-07-06T02:22:58.462455+0000","last_peered":"2023-07-06T02:22:58.462455+0000","last_clean":"2023-07-06T02:22:58.462455+0000","last_became_active":"2023-07-06T02:22:53.434127+0000","last_became_peered":"2023-07-06T02:22:53.434127+0000","last_unstale":"2023-07-06T02:22:58.462455+0000","last_undegraded":"2023-07-06T02:22:58.462455+0000","last_fullsized":"2023-07-06T02:22:58.462455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:36.857231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-07-06T02:22:58.496834+0000","last_change":"2023-07-06T02:22:53.433904+0000","last_active":"2023-07-06T02:22:58.496834+0000","last_peered":"2023-07-06T02:22:58.496834+0000","last_clean":"2023-07-06T02:22:58.496834+0000","last_became_active":"2023-07-06T02:22:53.433683+0000","last_became_peered":"2023-07-06T02:22:53.433683+0000","last_unstale":"2023-07-06T02:22:58.496834+0000","last_undegraded":"2023-07-06T02:22:58.496834+0000","last_fullsized":"2023-07-06T02:22:58.496834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:52.395900+0000","last_clean_scrub_stamp":"2023-07-06T02:22:52.395900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:34:34.629626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":699108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":108580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T02:23:18.348 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T02:23:18.348 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-06T02:23:18.349 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-06T02:23:18.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T02:23:18.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.497+0000 7fe776eea700 1 Processor -- start 2023-07-06T02:23:18.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.497+0000 7fe776eea700 1 -- start start 2023-07-06T02:23:18.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe776eea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe770156ab0 0x7fe770160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe776eea700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe7701600b0 0x7fe7700cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe776eea700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7700cb3d0 con 0x7fe770160bb0 2023-07-06T02:23:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe776eea700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe770156e80 con 0x7fe7701600b0 2023-07-06T02:23:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe776eea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700ca370 con 0x7fe770156ab0 2023-07-06T02:23:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe775487700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe7701600b0 0x7fe7700cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe76ffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe770160bb0 0x7fe7700b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57858/0 (socket says 172.21.15.89:57858) 2023-07-06T02:23:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.498+0000 7fe76ffff700 1 -- 172.21.15.89:0/859289931 learned_addr learned my addr 172.21.15.89:0/859289931 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:18.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe774c86700 1 --2- 172.21.15.89:0/859289931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe770156ab0 0x7fe770160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe775487700 1 -- 172.21.15.89:0/859289931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe770156ab0 msgr2=0x7fe770160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe775487700 1 --2- 172.21.15.89:0/859289931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe770156ab0 0x7fe770160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe775487700 1 -- 172.21.15.89:0/859289931 >> v1:172.21.15.89:6789/0 conn(0x7fe770160bb0 legacy=0x7fe7700b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe775487700 1 -- 172.21.15.89:0/859289931 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700ca610 con 0x7fe7701600b0 2023-07-06T02:23:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe76f7fe700 1 -- 172.21.15.89:0/859289931 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861464215 0 0) 0x7fe7700cb3d0 con 0x7fe770160bb0 2023-07-06T02:23:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe775487700 1 --2- 172.21.15.89:0/859289931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7701600b0 0x7fe7700cd700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0eac97948fb32e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe76f7fe700 1 -- 172.21.15.89:0/859289931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800e070 con 0x7fe7701600b0 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe76f7fe700 1 -- 172.21.15.89:0/859289931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76800bbd0 con 0x7fe7701600b0 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe76f7fe700 1 -- 172.21.15.89:0/859289931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768012430 con 0x7fe7701600b0 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe776eea700 1 -- 172.21.15.89:0/859289931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7701600b0 msgr2=0x7fe7700cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.499+0000 7fe776eea700 1 --2- 172.21.15.89:0/859289931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7701600b0 0x7fe7700cd700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.500+0000 7fe776eea700 1 -- 172.21.15.89:0/859289931 shutdown_connections 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.500+0000 7fe776eea700 1 --2- 172.21.15.89:0/859289931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7701600b0 0x7fe7700cd700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.500+0000 7fe776eea700 1 --2- 172.21.15.89:0/859289931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe770156ab0 0x7fe770160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.500+0000 7fe776eea700 1 -- 172.21.15.89:0/859289931 >> 172.21.15.89:0/859289931 conn(0x7fe7700cc490 msgr2=0x7fe7700cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.500+0000 7fe776eea700 1 -- 172.21.15.89:0/859289931 shutdown_connections 2023-07-06T02:23:18.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.500+0000 7fe776eea700 1 -- 172.21.15.89:0/859289931 wait complete. 2023-07-06T02:23:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.501+0000 7fe776eea700 1 Processor -- start 2023-07-06T02:23:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.501+0000 7fe776eea700 1 -- start start 2023-07-06T02:23:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770156ab0 0x7fe770155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7701600b0 0x7fe7700c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7700ad620 0x7fe7700ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c97a0 con 0x7fe770156ab0 2023-07-06T02:23:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700adfd0 con 0x7fe7700ad620 2023-07-06T02:23:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c77d0 con 0x7fe7701600b0 2023-07-06T02:23:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770156ab0 0x7fe770155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770156ab0 0x7fe770155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32962/0 (socket says 172.21.15.89:32962) 2023-07-06T02:23:18.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 -- 172.21.15.89:0/894636424 learned_addr learned my addr 172.21.15.89:0/894636424 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe775487700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7700ad620 0x7fe7700ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe76ffff700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7701600b0 0x7fe7700c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 -- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7701600b0 msgr2=0x7fe7700c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7701600b0 0x7fe7700c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 -- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7700ad620 msgr2=0x7fe7700ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7700ad620 0x7fe7700ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 -- 172.21.15.89:0/894636424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009a70 con 0x7fe770156ab0 2023-07-06T02:23:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe774c86700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770156ab0 0x7fe770155a80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e840 tx=0x7fe76400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe76d7fa700 1 -- 172.21.15.89:0/894636424 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764013610 con 0x7fe770156ab0 2023-07-06T02:23:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe76d7fa700 1 -- 172.21.15.89:0/894636424 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe764013770 con 0x7fe770156ab0 2023-07-06T02:23:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7701f2e50 con 0x7fe770156ab0 2023-07-06T02:23:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe76d7fa700 1 -- 172.21.15.89:0/894636424 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764013b20 con 0x7fe770156ab0 2023-07-06T02:23:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.502+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7700c7ce0 con 0x7fe770156ab0 2023-07-06T02:23:18.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.503+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe770074270 con 0x7fe770156ab0 2023-07-06T02:23:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.504+0000 7fe76d7fa700 1 -- 172.21.15.89:0/894636424 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe764013c80 con 0x7fe770156ab0 2023-07-06T02:23:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.504+0000 7fe76d7fa700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe75406ace0 0x7fe75406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.504+0000 7fe76d7fa700 1 -- 172.21.15.89:0/894636424 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe7640892c0 con 0x7fe770156ab0 2023-07-06T02:23:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.504+0000 7fe76ffff700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe75406ace0 0x7fe75406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.504+0000 7fe76ffff700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe75406ace0 0x7fe75406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b5960 tx=0x7fe75c007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.508+0000 7fe76d7fa700 1 -- 172.21.15.89:0/894636424 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe764054090 con 0x7fe770156ab0 2023-07-06T02:23:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.740+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe7700b5990 con 0x7fe770156ab0 2023-07-06T02:23:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.740+0000 7fe76d7fa700 1 -- 172.21.15.89:0/894636424 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe7640576b0 con 0x7fe770156ab0 2023-07-06T02:23:18.744 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:18.744 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T02:23:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe75406ace0 msgr2=0x7fe75406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe75406ace0 0x7fe75406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b5960 tx=0x7fe75c007e70 comp rx=0 tx=0).stop 2023-07-06T02:23:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770156ab0 msgr2=0x7fe770155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770156ab0 0x7fe770155a80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e840 tx=0x7fe76400eb50 comp rx=0 tx=0).stop 2023-07-06T02:23:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 shutdown_connections 2023-07-06T02:23:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7700ad620 0x7fe7700ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe75406ace0 0x7fe75406d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770156ab0 0x7fe770155a80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 --2- 172.21.15.89:0/894636424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7701600b0 0x7fe7700c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 >> 172.21.15.89:0/894636424 conn(0x7fe7700cc490 msgr2=0x7fe7700ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 shutdown_connections 2023-07-06T02:23:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.743+0000 7fe776eea700 1 -- 172.21.15.89:0/894636424 wait complete. 2023-07-06T02:23:18.757 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-06T02:23:18.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:18.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 Processor -- start 2023-07-06T02:23:18.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 -- start start 2023-07-06T02:23:18.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0fc415a0c0 0x7f0fc40cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc4160a00 0x7f0fc4070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0fc4070870 0x7f0fc40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc415a490 con 0x7f0fc4070870 2023-07-06T02:23:18.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc4070cd0 con 0x7f0fc415a0c0 2023-07-06T02:23:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.875+0000 7f0fc9a31700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40d04b0 con 0x7f0fc4160a00 2023-07-06T02:23:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0fc4070870 0x7f0fc40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0fc4070870 0x7f0fc40ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32972/0 (socket says 172.21.15.89:32972) 2023-07-06T02:23:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 -- 172.21.15.89:0/3969664796 learned_addr learned my addr 172.21.15.89:0/3969664796 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc2ffd700 1 --2- 172.21.15.89:0/3969664796 >> v2:172.21.15.172:3300/0 conn(0x7f0fc415a0c0 0x7f0fc40cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 -- 172.21.15.89:0/3969664796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc4160a00 msgr2=0x7f0fc4070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 --2- 172.21.15.89:0/3969664796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc4160a00 0x7f0fc4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 -- 172.21.15.89:0/3969664796 >> v2:172.21.15.172:3300/0 conn(0x7f0fc415a0c0 msgr2=0x7f0fc40cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 --2- 172.21.15.89:0/3969664796 >> v2:172.21.15.172:3300/0 conn(0x7f0fc415a0c0 0x7f0fc40cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 -- 172.21.15.89:0/3969664796 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc415c9b0 con 0x7f0fc4070870 2023-07-06T02:23:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc37fe700 1 --2- 172.21.15.89:0/3969664796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4070870 0x7f0fc40ce6d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4072270 tx=0x7f0fbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0c3f6a2000a25e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc1ffb700 1 -- 172.21.15.89:0/3969664796 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00e070 con 0x7f0fc4070870 2023-07-06T02:23:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc1ffb700 1 -- 172.21.15.89:0/3969664796 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc00b9c0 con 0x7f0fc4070870 2023-07-06T02:23:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.876+0000 7f0fc1ffb700 1 -- 172.21.15.89:0/3969664796 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00bd70 con 0x7f0fc4070870 2023-07-06T02:23:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 -- 172.21.15.89:0/3969664796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4070870 msgr2=0x7f0fc40ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/3969664796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4070870 0x7f0fc40ce6d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4072270 tx=0x7f0fbc00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 -- 172.21.15.89:0/3969664796 shutdown_connections 2023-07-06T02:23:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/3969664796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4070870 0x7f0fc40ce6d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/3969664796 >> v2:172.21.15.172:3300/0 conn(0x7f0fc415a0c0 0x7f0fc40cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/3969664796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc4160a00 0x7f0fc4070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 -- 172.21.15.89:0/3969664796 >> 172.21.15.89:0/3969664796 conn(0x7f0fc40d0b00 msgr2=0x7f0fc40d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:18.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 -- 172.21.15.89:0/3969664796 shutdown_connections 2023-07-06T02:23:18.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.877+0000 7f0fc9a31700 1 -- 172.21.15.89:0/3969664796 wait complete. 2023-07-06T02:23:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.878+0000 7f0fc9a31700 1 Processor -- start 2023-07-06T02:23:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.878+0000 7f0fc9a31700 1 -- start start 2023-07-06T02:23:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc9a31700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc4070870 0x7f0fc4152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc9a31700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc415a0c0 0x7f0fc41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc9a31700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4160a00 0x7f0fc415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc9a31700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4156f40 con 0x7f0fc4160a00 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc9a31700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4152910 con 0x7f0fc4070870 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc9a31700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4156c60 con 0x7f0fc415a0c0 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4160a00 0x7f0fc415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4160a00 0x7f0fc415ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32980/0 (socket says 172.21.15.89:32980) 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 -- 172.21.15.89:0/769917936 learned_addr learned my addr 172.21.15.89:0/769917936 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc2ffd700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc4070870 0x7f0fc4152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc27fc700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc415a0c0 0x7f0fc41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 -- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc415a0c0 msgr2=0x7f0fc41523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc415a0c0 0x7f0fc41523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 -- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc4070870 msgr2=0x7f0fc4152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc4070870 0x7f0fc4152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 -- 172.21.15.89:0/769917936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc009b30 con 0x7f0fc4160a00 2023-07-06T02:23:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc27fc700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc415a0c0 0x7f0fc41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fc37fe700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4160a00 0x7f0fc415ebf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b370 tx=0x7f0fbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.879+0000 7f0fabfff700 1 -- 172.21.15.89:0/769917936 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004310 con 0x7f0fc4160a00 2023-07-06T02:23:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.880+0000 7f0fabfff700 1 -- 172.21.15.89:0/769917936 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc004470 con 0x7f0fc4160a00 2023-07-06T02:23:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.880+0000 7f0fabfff700 1 -- 172.21.15.89:0/769917936 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004820 con 0x7f0fc4160a00 2023-07-06T02:23:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.880+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc40c5a50 con 0x7f0fc4160a00 2023-07-06T02:23:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.880+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc40c6010 con 0x7f0fc4160a00 2023-07-06T02:23:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.880+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc4074270 con 0x7f0fc4160a00 2023-07-06T02:23:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.881+0000 7f0fabfff700 1 -- 172.21.15.89:0/769917936 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fbc01c440 con 0x7f0fc4160a00 2023-07-06T02:23:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.881+0000 7f0fabfff700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0fb006ace0 0x7f0fb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.881+0000 7f0fabfff700 1 -- 172.21.15.89:0/769917936 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0fbc08afc0 con 0x7f0fc4160a00 2023-07-06T02:23:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.881+0000 7f0fc2ffd700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0fb006ace0 0x7f0fb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.882+0000 7f0fc2ffd700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0fb006ace0 0x7f0fb006d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40cd100 tx=0x7f0fb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:18.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:18.886+0000 7f0fabfff700 1 -- 172.21.15.89:0/769917936 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fbc0bd310 con 0x7f0fc4160a00 2023-07-06T02:23:19.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.072+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fc40c4110 con 0x7f0fc4160a00 2023-07-06T02:23:19.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.073+0000 7f0fabfff700 1 -- 172.21.15.89:0/769917936 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f0fbc020090 con 0x7f0fc4160a00 2023-07-06T02:23:19.076 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:19.077 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:19.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0fb006ace0 msgr2=0x7f0fb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0fb006ace0 0x7f0fb006d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40cd100 tx=0x7f0fb4008040 comp rx=0 tx=0).stop 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4160a00 msgr2=0x7f0fc415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4160a00 0x7f0fc415ebf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b370 tx=0x7f0fbc004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 shutdown_connections 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0fb006ace0 0x7f0fb006d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fc4160a00 0x7f0fc415ebf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc4070870 0x7f0fc4152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 --2- 172.21.15.89:0/769917936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc415a0c0 0x7f0fc41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 >> 172.21.15.89:0/769917936 conn(0x7f0fc40d0b00 msgr2=0x7f0fc40b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:19.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 shutdown_connections 2023-07-06T02:23:19.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.075+0000 7f0fc9a31700 1 -- 172.21.15.89:0/769917936 wait complete. 2023-07-06T02:23:19.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:19.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:19.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.213+0000 7fb086d09700 1 Processor -- start 2023-07-06T02:23:19.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.213+0000 7fb086d09700 1 -- start start 2023-07-06T02:23:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb086d09700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb080070f20 0x7fb0800713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb086d09700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb080160000 0x7fb080160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb086d09700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0800cb9f0 con 0x7fb08015c500 2023-07-06T02:23:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb086d09700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb080071900 con 0x7fb080070f20 2023-07-06T02:23:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb086d09700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08015d8e0 con 0x7fb080160000 2023-07-06T02:23:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb07ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb080070f20 0x7fb0800713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb07ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb080070f20 0x7fb0800713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32986/0 (socket says 172.21.15.89:32986) 2023-07-06T02:23:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb07ffff700 1 -- 172.21.15.89:0/3181975045 learned_addr learned my addr 172.21.15.89:0/3181975045 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb0852a6700 1 --2- 172.21.15.89:0/3181975045 >> v2:172.21.15.172:3300/0 conn(0x7fb080160000 0x7fb080160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb07ffff700 1 -- 172.21.15.89:0/3181975045 >> v2:172.21.15.172:3300/0 conn(0x7fb080160000 msgr2=0x7fb080160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb07ffff700 1 --2- 172.21.15.89:0/3181975045 >> v2:172.21.15.172:3300/0 conn(0x7fb080160000 0x7fb080160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb07ffff700 1 -- 172.21.15.89:0/3181975045 >> v1:172.21.15.172:6789/0 conn(0x7fb08015c500 legacy=0x7fb08006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.214+0000 7fb07ffff700 1 -- 172.21.15.89:0/3181975045 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb080160960 con 0x7fb080070f20 2023-07-06T02:23:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb07ffff700 1 --2- 172.21.15.89:0/3181975045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0800713c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb08015db40 tx=0x7fb06c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfb820dde432fa4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb07f7fe700 1 -- 172.21.15.89:0/3181975045 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c00b870 con 0x7fb080070f20 2023-07-06T02:23:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb07f7fe700 1 -- 172.21.15.89:0/3181975045 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c00b9d0 con 0x7fb080070f20 2023-07-06T02:23:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb07f7fe700 1 -- 172.21.15.89:0/3181975045 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c00bd60 con 0x7fb080070f20 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 -- 172.21.15.89:0/3181975045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 msgr2=0x7fb0800713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 --2- 172.21.15.89:0/3181975045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0800713c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb08015db40 tx=0x7fb06c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 -- 172.21.15.89:0/3181975045 shutdown_connections 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 --2- 172.21.15.89:0/3181975045 >> v2:172.21.15.172:3300/0 conn(0x7fb080160000 0x7fb080160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 --2- 172.21.15.89:0/3181975045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0800713c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 -- 172.21.15.89:0/3181975045 >> 172.21.15.89:0/3181975045 conn(0x7fb0800cf1c0 msgr2=0x7fb0800cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 -- 172.21.15.89:0/3181975045 shutdown_connections 2023-07-06T02:23:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.215+0000 7fb086d09700 1 -- 172.21.15.89:0/3181975045 wait complete. 2023-07-06T02:23:19.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 Processor -- start 2023-07-06T02:23:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 -- start start 2023-07-06T02:23:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0801503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb08015c500 0x7fb08014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb080160000 0x7fb0800ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0800af040 con 0x7fb080070f20 2023-07-06T02:23:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0800c5ba0 con 0x7fb080160000 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.217+0000 7fb086d09700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0800c55e0 con 0x7fb08015c500 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0801503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0801503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32990/0 (socket says 172.21.15.89:32990) 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 -- 172.21.15.89:0/1251058926 learned_addr learned my addr 172.21.15.89:0/1251058926 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb07ffff700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb08015c500 0x7fb08014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb0852a6700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb080160000 0x7fb0800ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 -- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb08015c500 msgr2=0x7fb08014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb08015c500 0x7fb08014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 -- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb080160000 msgr2=0x7fb0800ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb080160000 0x7fb0800ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 -- 172.21.15.89:0/1251058926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c009550 con 0x7fb080070f20 2023-07-06T02:23:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb084aa5700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0801503d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb07400c4e0 tx=0x7fb07400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb07d7fa700 1 -- 172.21.15.89:0/1251058926 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740052d0 con 0x7fb080070f20 2023-07-06T02:23:19.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb07d7fa700 1 -- 172.21.15.89:0/1251058926 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb074005430 con 0x7fb080070f20 2023-07-06T02:23:19.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb07d7fa700 1 -- 172.21.15.89:0/1251058926 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740038c0 con 0x7fb080070f20 2023-07-06T02:23:19.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0800c58c0 con 0x7fb080070f20 2023-07-06T02:23:19.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.218+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0801500f0 con 0x7fb080070f20 2023-07-06T02:23:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.219+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb080066fb0 con 0x7fb080070f20 2023-07-06T02:23:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.219+0000 7fb07d7fa700 1 -- 172.21.15.89:0/1251058926 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb074003a20 con 0x7fb080070f20 2023-07-06T02:23:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.220+0000 7fb07d7fa700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb06406ace0 0x7fb06406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.220+0000 7fb07ffff700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb06406ace0 0x7fb06406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.220+0000 7fb07d7fa700 1 -- 172.21.15.89:0/1251058926 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb074088c00 con 0x7fb080070f20 2023-07-06T02:23:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.220+0000 7fb07ffff700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb06406ace0 0x7fb06406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb06c000c00 tx=0x7fb06c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.225+0000 7fb07d7fa700 1 -- 172.21.15.89:0/1251058926 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb07408f080 con 0x7fb080070f20 2023-07-06T02:23:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.412+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb080074270 con 0x7fb080070f20 2023-07-06T02:23:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.412+0000 7fb07d7fa700 1 -- 172.21.15.89:0/1251058926 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb074017020 con 0x7fb080070f20 2023-07-06T02:23:19.416 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb06406ace0 msgr2=0x7fb06406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb06406ace0 0x7fb06406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb06c000c00 tx=0x7fb06c008030 comp rx=0 tx=0).stop 2023-07-06T02:23:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 msgr2=0x7fb0801503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0801503d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb07400c4e0 tx=0x7fb07400c8a0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 shutdown_connections 2023-07-06T02:23:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb080160000 0x7fb0800ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb06406ace0 0x7fb06406d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb080070f20 0x7fb0801503d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 --2- 172.21.15.89:0/1251058926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb08015c500 0x7fb08014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 >> 172.21.15.89:0/1251058926 conn(0x7fb0800cf1c0 msgr2=0x7fb08006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 shutdown_connections 2023-07-06T02:23:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.415+0000 7fb086d09700 1 -- 172.21.15.89:0/1251058926 wait complete. 2023-07-06T02:23:19.430 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:19.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.551+0000 7f36b7973700 1 Processor -- start 2023-07-06T02:23:19.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.551+0000 7f36b7973700 1 -- start start 2023-07-06T02:23:19.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b7973700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f36b0156ab0 0x7f36b0160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b7973700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b0160bb0 0x7f36b00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b7973700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b00cb3d0 con 0x7f36b01600b0 2023-07-06T02:23:19.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b7973700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b0156e80 con 0x7f36b0156ab0 2023-07-06T02:23:19.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b7973700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00ca370 con 0x7f36b0160bb0 2023-07-06T02:23:19.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b570f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f36b0156ab0 0x7f36b0160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b4f0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b0160bb0 0x7f36b00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b570f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f36b0156ab0 0x7f36b0160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:59988/0 (socket says 172.21.15.89:59988) 2023-07-06T02:23:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b4f0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b0160bb0 0x7f36b00b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:56206/0 (socket says 172.21.15.89:56206) 2023-07-06T02:23:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b5f10700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f36b01600b0 0x7f36b006d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33660/0 (socket says 172.21.15.89:33660) 2023-07-06T02:23:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.552+0000 7f36b570f700 1 -- 172.21.15.89:0/1146757278 learned_addr learned my addr 172.21.15.89:0/1146757278 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36b570f700 1 -- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b0160bb0 msgr2=0x7f36b00b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36b570f700 1 --2- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b0160bb0 0x7f36b00b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36b570f700 1 -- 172.21.15.89:0/1146757278 >> v1:172.21.15.172:6789/0 conn(0x7f36b01600b0 legacy=0x7f36b006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36b570f700 1 -- 172.21.15.89:0/1146757278 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b00ca610 con 0x7f36b0156ab0 2023-07-06T02:23:19.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36b4f0e700 1 --2- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b0160bb0 0x7f36b00b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:19.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36b570f700 1 --2- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 0x7f36b0160670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36a00098f0 tx=0x7f36a0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c7a209c1a0aa3a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36a7fff700 1 -- 172.21.15.89:0/1146757278 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0011830 con 0x7f36b0156ab0 2023-07-06T02:23:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36a7fff700 1 -- 172.21.15.89:0/1146757278 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0011990 con 0x7f36b0156ab0 2023-07-06T02:23:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.553+0000 7f36a7fff700 1 -- 172.21.15.89:0/1146757278 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0011d20 con 0x7f36b0156ab0 2023-07-06T02:23:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 -- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 msgr2=0x7f36b0160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 --2- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 0x7f36b0160670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36a00098f0 tx=0x7f36a0009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 -- 172.21.15.89:0/1146757278 shutdown_connections 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 --2- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 0x7f36b0160670 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 --2- 172.21.15.89:0/1146757278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b0160bb0 0x7f36b00b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 -- 172.21.15.89:0/1146757278 >> 172.21.15.89:0/1146757278 conn(0x7f36b00cc490 msgr2=0x7f36b00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 -- 172.21.15.89:0/1146757278 shutdown_connections 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.554+0000 7f36b7973700 1 -- 172.21.15.89:0/1146757278 wait complete. 2023-07-06T02:23:19.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.555+0000 7f36b7973700 1 Processor -- start 2023-07-06T02:23:19.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.555+0000 7f36b7973700 1 -- start start 2023-07-06T02:23:19.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b7973700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 0x7f36b00ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b7973700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36b01600b0 0x7f36b00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b7973700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b00c3ba0 0x7f36b00c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b7973700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0145840 con 0x7f36b01600b0 2023-07-06T02:23:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b7973700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00c4010 con 0x7f36b0156ab0 2023-07-06T02:23:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b7973700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00ad020 con 0x7f36b00c3ba0 2023-07-06T02:23:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b4f0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36b01600b0 0x7f36b00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b4f0e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36b01600b0 0x7f36b00c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33006/0 (socket says 172.21.15.89:33006) 2023-07-06T02:23:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b4f0e700 1 -- 172.21.15.89:0/1124460282 learned_addr learned my addr 172.21.15.89:0/1124460282 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b570f700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 0x7f36b00ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.556+0000 7f36b5f10700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b00c3ba0 0x7f36b00c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b4f0e700 1 -- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b00c3ba0 msgr2=0x7f36b00c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b4f0e700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b00c3ba0 0x7f36b00c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b4f0e700 1 -- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 msgr2=0x7f36b00ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b4f0e700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 0x7f36b00ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b4f0e700 1 -- 172.21.15.89:0/1124460282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0094a0 con 0x7f36b01600b0 2023-07-06T02:23:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b4f0e700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36b01600b0 0x7f36b00c3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00ebe0 tx=0x7f36ac00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36a5ffb700 1 -- 172.21.15.89:0/1124460282 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac009760 con 0x7f36b01600b0 2023-07-06T02:23:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36a5ffb700 1 -- 172.21.15.89:0/1124460282 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00c850 con 0x7f36b01600b0 2023-07-06T02:23:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a0009550 con 0x7f36b01600b0 2023-07-06T02:23:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36a5ffb700 1 -- 172.21.15.89:0/1124460282 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00cbe0 con 0x7f36b01600b0 2023-07-06T02:23:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.557+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b00ad5e0 con 0x7f36b01600b0 2023-07-06T02:23:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.558+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0074270 con 0x7f36b01600b0 2023-07-06T02:23:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.559+0000 7f36a5ffb700 1 -- 172.21.15.89:0/1124460282 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36ac00cd40 con 0x7f36b01600b0 2023-07-06T02:23:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.559+0000 7f36a5ffb700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f369406ace0 0x7f369406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.559+0000 7f36a5ffb700 1 -- 172.21.15.89:0/1124460282 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f36ac0896a0 con 0x7f36b01600b0 2023-07-06T02:23:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.559+0000 7f36b570f700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f369406ace0 0x7f369406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.559+0000 7f36b570f700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f369406ace0 0x7f369406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009c40 tx=0x7f36a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.563+0000 7f36a5ffb700 1 -- 172.21.15.89:0/1124460282 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f36ac0544f0 con 0x7f36b01600b0 2023-07-06T02:23:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.753+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36b015d6f0 con 0x7f36b01600b0 2023-07-06T02:23:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.753+0000 7f36a5ffb700 1 -- 172.21.15.89:0/1124460282 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36ac057b10 con 0x7f36b01600b0 2023-07-06T02:23:19.757 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f369406ace0 msgr2=0x7f369406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f369406ace0 0x7f369406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009c40 tx=0x7f36a0008030 comp rx=0 tx=0).stop 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36b01600b0 msgr2=0x7f36b00c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36b01600b0 0x7f36b00c3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00ebe0 tx=0x7f36ac00eef0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 shutdown_connections 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36b00c3ba0 0x7f36b00c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36b0156ab0 0x7f36b00ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f369406ace0 0x7f369406d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 --2- 172.21.15.89:0/1124460282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36b01600b0 0x7f36b00c3660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 >> 172.21.15.89:0/1124460282 conn(0x7f36b00cc490 msgr2=0x7f36b006b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:19.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 shutdown_connections 2023-07-06T02:23:19.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.755+0000 7f36b7973700 1 -- 172.21.15.89:0/1124460282 wait complete. 2023-07-06T02:23:19.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.890+0000 7fb3192d9700 1 Processor -- start 2023-07-06T02:23:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.890+0000 7fb3192d9700 1 -- start start 2023-07-06T02:23:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.890+0000 7fb3192d9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb3140ae4e0 0x7fb31406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.890+0000 7fb3192d9700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb314160b50 con 0x7fb314071080 2023-07-06T02:23:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.890+0000 7fb3192d9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb314160e30 con 0x7fb314071450 2023-07-06T02:23:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.890+0000 7fb3192d9700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31406de90 con 0x7fb3140ae4e0 2023-07-06T02:23:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.890+0000 7fb3137fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb3140ae4e0 0x7fb31406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb312ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb314071080 0x7fb314160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57914/0 (socket says 172.21.15.89:57914) 2023-07-06T02:23:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb312ffd700 1 -- 172.21.15.89:0/1224052034 learned_addr learned my addr 172.21.15.89:0/1224052034 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb3137fe700 1 -- 172.21.15.89:0/1224052034 >> v1:172.21.15.172:6789/0 conn(0x7fb314071450 legacy=0x7fb3140addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb3137fe700 1 -- 172.21.15.89:0/1224052034 >> v1:172.21.15.89:6789/0 conn(0x7fb314071080 legacy=0x7fb314160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb3137fe700 1 -- 172.21.15.89:0/1224052034 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31406ce70 con 0x7fb3140ae4e0 2023-07-06T02:23:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb311ffb700 1 -- 172.21.15.89:0/1224052034 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127516696 0 0) 0x7fb314160b50 con 0x7fb314071080 2023-07-06T02:23:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb3137fe700 1 --2- 172.21.15.89:0/1224052034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb31406d950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009d90 tx=0x7fb30c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781147eb8cce0ffa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb311ffb700 1 -- 172.21.15.89:0/1224052034 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00e070 con 0x7fb3140ae4e0 2023-07-06T02:23:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb311ffb700 1 -- 172.21.15.89:0/1224052034 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00bb90 con 0x7fb3140ae4e0 2023-07-06T02:23:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.891+0000 7fb311ffb700 1 -- 172.21.15.89:0/1224052034 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c003a20 con 0x7fb3140ae4e0 2023-07-06T02:23:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.892+0000 7fb3192d9700 1 -- 172.21.15.89:0/1224052034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 msgr2=0x7fb31406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.892+0000 7fb3192d9700 1 --2- 172.21.15.89:0/1224052034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb31406d950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009d90 tx=0x7fb30c00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.892+0000 7fb3192d9700 1 -- 172.21.15.89:0/1224052034 shutdown_connections 2023-07-06T02:23:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.892+0000 7fb3192d9700 1 --2- 172.21.15.89:0/1224052034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb31406d950 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.892+0000 7fb3192d9700 1 -- 172.21.15.89:0/1224052034 >> 172.21.15.89:0/1224052034 conn(0x7fb3140cf0f0 msgr2=0x7fb3140cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.892+0000 7fb3192d9700 1 -- 172.21.15.89:0/1224052034 shutdown_connections 2023-07-06T02:23:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.892+0000 7fb3192d9700 1 -- 172.21.15.89:0/1224052034 wait complete. 2023-07-06T02:23:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.893+0000 7fb3192d9700 1 Processor -- start 2023-07-06T02:23:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.893+0000 7fb3192d9700 1 -- start start 2023-07-06T02:23:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb314071080 0x7fb314156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb314071450 0x7fb3141e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb3141e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314157220 con 0x7fb3140ae4e0 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3141e8170 con 0x7fb314071450 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b13d0 con 0x7fb314071080 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb3141e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb3141e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33022/0 (socket says 172.21.15.89:33022) 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 -- 172.21.15.89:0/2196543184 learned_addr learned my addr 172.21.15.89:0/2196543184 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3127fc700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb314071450 0x7fb3141e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb312ffd700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb314071080 0x7fb314156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 -- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb314071080 msgr2=0x7fb314156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb314071080 0x7fb314156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 -- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb314071450 msgr2=0x7fb3141e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb314071450 0x7fb3141e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 -- 172.21.15.89:0/2196543184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c009a70 con 0x7fb3140ae4e0 2023-07-06T02:23:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3137fe700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb3141e7150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb31409d100 tx=0x7fb30c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb2fb7fe700 1 -- 172.21.15.89:0/2196543184 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c012630 con 0x7fb3140ae4e0 2023-07-06T02:23:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb2fb7fe700 1 -- 172.21.15.89:0/2196543184 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c0127b0 con 0x7fb3140ae4e0 2023-07-06T02:23:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3141e7b20 con 0x7fb3140ae4e0 2023-07-06T02:23:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb2fb7fe700 1 -- 172.21.15.89:0/2196543184 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c012b60 con 0x7fb3140ae4e0 2023-07-06T02:23:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.894+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb314157b00 con 0x7fb3140ae4e0 2023-07-06T02:23:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.895+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3141e82b0 con 0x7fb3140ae4e0 2023-07-06T02:23:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.896+0000 7fb2fb7fe700 1 -- 172.21.15.89:0/2196543184 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb30c012cc0 con 0x7fb3140ae4e0 2023-07-06T02:23:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.896+0000 7fb2fb7fe700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb30006ace0 0x7fb30006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.896+0000 7fb2fb7fe700 1 -- 172.21.15.89:0/2196543184 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb30c08b240 con 0x7fb3140ae4e0 2023-07-06T02:23:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.896+0000 7fb312ffd700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb30006ace0 0x7fb30006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.897+0000 7fb312ffd700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb30006ace0 0x7fb30006d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb304000f60 tx=0x7fb304007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:19.900+0000 7fb2fb7fe700 1 -- 172.21.15.89:0/2196543184 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb30c056090 con 0x7fb3140ae4e0 2023-07-06T02:23:20.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.088+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb314074270 con 0x7fb3140ae4e0 2023-07-06T02:23:20.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.089+0000 7fb2fb7fe700 1 -- 172.21.15.89:0/2196543184 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb30c0596b0 con 0x7fb3140ae4e0 2023-07-06T02:23:20.092 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:20.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb30006ace0 msgr2=0x7fb30006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb30006ace0 0x7fb30006d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb304000f60 tx=0x7fb304007be0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 msgr2=0x7fb3141e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb3141e7150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb31409d100 tx=0x7fb30c004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 shutdown_connections 2023-07-06T02:23:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb30006ace0 0x7fb30006d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3140ae4e0 0x7fb3141e7150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb314071080 0x7fb314156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 --2- 172.21.15.89:0/2196543184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb314071450 0x7fb3141e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 >> 172.21.15.89:0/2196543184 conn(0x7fb3140cf0f0 msgr2=0x7fb3140cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 shutdown_connections 2023-07-06T02:23:20.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.091+0000 7fb3192d9700 1 -- 172.21.15.89:0/2196543184 wait complete. 2023-07-06T02:23:20.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:20.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.226+0000 7f443f126700 1 Processor -- start 2023-07-06T02:23:20.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.226+0000 7f443f126700 1 -- start start 2023-07-06T02:23:20.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443f126700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f44380c9a60 0x7f443806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443f126700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4438071630 0x7f443806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443f126700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443806f290 con 0x7f443806e250 2023-07-06T02:23:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443f126700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443806ecd0 con 0x7f44380c9a60 2023-07-06T02:23:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443f126700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443806efb0 con 0x7f4438071630 2023-07-06T02:23:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443cec2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f44380c9a60 0x7f443806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443cec2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f44380c9a60 0x7f443806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33026/0 (socket says 172.21.15.89:33026) 2023-07-06T02:23:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443cec2700 1 -- 172.21.15.89:0/3656006899 learned_addr learned my addr 172.21.15.89:0/3656006899 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443d6c3700 1 --2- 172.21.15.89:0/3656006899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4438071630 0x7f443806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443cec2700 1 -- 172.21.15.89:0/3656006899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4438071630 msgr2=0x7f443806e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443cec2700 1 --2- 172.21.15.89:0/3656006899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4438071630 0x7f443806e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443cec2700 1 -- 172.21.15.89:0/3656006899 >> v1:172.21.15.172:6789/0 conn(0x7f443806e250 legacy=0x7f4438070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443cec2700 1 -- 172.21.15.89:0/3656006899 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443806a990 con 0x7f44380c9a60 2023-07-06T02:23:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.227+0000 7f443d6c3700 1 --2- 172.21.15.89:0/3656006899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4438071630 0x7f443806e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443cec2700 1 --2- 172.21.15.89:0/3656006899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44380c9a60 0x7f443806a060 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f442c00bb90 tx=0x7f442c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e5f6d923ce104 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f44377fe700 1 -- 172.21.15.89:0/3656006899 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00dd50 con 0x7f44380c9a60 2023-07-06T02:23:20.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f44377fe700 1 -- 172.21.15.89:0/3656006899 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c010470 con 0x7f44380c9a60 2023-07-06T02:23:20.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f44377fe700 1 -- 172.21.15.89:0/3656006899 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c010800 con 0x7f44380c9a60 2023-07-06T02:23:20.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 -- 172.21.15.89:0/3656006899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44380c9a60 msgr2=0x7f443806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 --2- 172.21.15.89:0/3656006899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44380c9a60 0x7f443806a060 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f442c00bb90 tx=0x7f442c00bea0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 -- 172.21.15.89:0/3656006899 shutdown_connections 2023-07-06T02:23:20.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 --2- 172.21.15.89:0/3656006899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4438071630 0x7f443806e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 --2- 172.21.15.89:0/3656006899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44380c9a60 0x7f443806a060 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 -- 172.21.15.89:0/3656006899 >> 172.21.15.89:0/3656006899 conn(0x7f44380cf1c0 msgr2=0x7f44380cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:20.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 -- 172.21.15.89:0/3656006899 shutdown_connections 2023-07-06T02:23:20.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.228+0000 7f443f126700 1 -- 172.21.15.89:0/3656006899 wait complete. 2023-07-06T02:23:20.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.229+0000 7f443f126700 1 Processor -- start 2023-07-06T02:23:20.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f443f126700 1 -- start start 2023-07-06T02:23:20.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f443f126700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443806e250 0x7f4438152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f443f126700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4438071630 0x7f443815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f443f126700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44380c9a60 0x7f44380c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f443f126700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443815a950 con 0x7f4438071630 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f443f126700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c8f80 con 0x7f44380c9a60 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f443f126700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4438152b40 con 0x7f443806e250 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f4437fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4438071630 0x7f443815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.230+0000 7f4437fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4438071630 0x7f443815b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33032/0 (socket says 172.21.15.89:33032) 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f4437fff700 1 -- 172.21.15.89:0/2813431387 learned_addr learned my addr 172.21.15.89:0/2813431387 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:20.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f443cec2700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443806e250 0x7f4438152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f443d6c3700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44380c9a60 0x7f44380c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f4437fff700 1 -- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443806e250 msgr2=0x7f4438152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f4437fff700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443806e250 0x7f4438152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f4437fff700 1 -- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44380c9a60 msgr2=0x7f44380c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f4437fff700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44380c9a60 0x7f44380c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f4437fff700 1 -- 172.21.15.89:0/2813431387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c00b7f0 con 0x7f4438071630 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f4437fff700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4438071630 0x7f443815b3a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f442400c4e0 tx=0x7f442400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f44357fa700 1 -- 172.21.15.89:0/2813431387 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240052d0 con 0x7f4438071630 2023-07-06T02:23:20.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f44357fa700 1 -- 172.21.15.89:0/2813431387 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424005430 con 0x7f4438071630 2023-07-06T02:23:20.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f44357fa700 1 -- 172.21.15.89:0/2813431387 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240038c0 con 0x7f4438071630 2023-07-06T02:23:20.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44380c7f80 con 0x7f4438071630 2023-07-06T02:23:20.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.231+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443815aba0 con 0x7f4438071630 2023-07-06T02:23:20.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.232+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4438074270 con 0x7f4438071630 2023-07-06T02:23:20.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.233+0000 7f44357fa700 1 -- 172.21.15.89:0/2813431387 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4424003a20 con 0x7f4438071630 2023-07-06T02:23:20.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.233+0000 7f44357fa700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f441c06ad10 0x7f441c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.233+0000 7f44357fa700 1 -- 172.21.15.89:0/2813431387 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4424088ae0 con 0x7f4438071630 2023-07-06T02:23:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.233+0000 7f443cec2700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f441c06ad10 0x7f441c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.234+0000 7f443cec2700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f441c06ad10 0x7f441c06d1c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f442c00bee0 tx=0x7f442c013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.238+0000 7f44357fa700 1 -- 172.21.15.89:0/2813431387 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f44240538a0 con 0x7f4438071630 2023-07-06T02:23:20.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.425+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44380b1b50 con 0x7f4438071630 2023-07-06T02:23:20.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.426+0000 7f44357fa700 1 -- 172.21.15.89:0/2813431387 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4424056ec0 con 0x7f4438071630 2023-07-06T02:23:20.429 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:20.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f441c06ad10 msgr2=0x7f441c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f441c06ad10 0x7f441c06d1c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f442c00bee0 tx=0x7f442c013000 comp rx=0 tx=0).stop 2023-07-06T02:23:20.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4438071630 msgr2=0x7f443815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4438071630 0x7f443815b3a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f442400c4e0 tx=0x7f442400c7f0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 shutdown_connections 2023-07-06T02:23:20.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44380c9a60 0x7f44380c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443806e250 0x7f4438152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f441c06ad10 0x7f441c06d1c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 --2- 172.21.15.89:0/2813431387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4438071630 0x7f443815b3a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 >> 172.21.15.89:0/2813431387 conn(0x7f44380cf1c0 msgr2=0x7f44380b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:20.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 shutdown_connections 2023-07-06T02:23:20.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.428+0000 7f443f126700 1 -- 172.21.15.89:0/2813431387 wait complete. 2023-07-06T02:23:20.443 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:20.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.563+0000 7f407e77c700 1 Processor -- start 2023-07-06T02:23:20.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.563+0000 7f407e77c700 1 -- start start 2023-07-06T02:23:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.563+0000 7f407e77c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4078070f20 0x7f40780c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.563+0000 7f407e77c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40780712f0 0x7f407806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.563+0000 7f407e77c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f407806e790 0x7f407806b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407e77c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780c9fa0 con 0x7f407806e790 2023-07-06T02:23:20.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407e77c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407806f290 con 0x7f4078070f20 2023-07-06T02:23:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407e77c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407806ecd0 con 0x7f40780712f0 2023-07-06T02:23:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f407806e790 0x7f407806b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f407806e790 0x7f407806b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33042/0 (socket says 172.21.15.89:33042) 2023-07-06T02:23:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 -- 172.21.15.89:0/2711389963 learned_addr learned my addr 172.21.15.89:0/2711389963 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f4077fff700 1 --2- 172.21.15.89:0/2711389963 >> v2:172.21.15.172:3300/0 conn(0x7f4078070f20 0x7f40780c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f40777fe700 1 --2- 172.21.15.89:0/2711389963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40780712f0 0x7f407806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 -- 172.21.15.89:0/2711389963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40780712f0 msgr2=0x7f407806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 --2- 172.21.15.89:0/2711389963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40780712f0 0x7f407806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 -- 172.21.15.89:0/2711389963 >> v2:172.21.15.172:3300/0 conn(0x7f4078070f20 msgr2=0x7f40780c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 --2- 172.21.15.89:0/2711389963 >> v2:172.21.15.172:3300/0 conn(0x7f4078070f20 0x7f40780c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 -- 172.21.15.89:0/2711389963 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407806efb0 con 0x7f407806e790 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f4077fff700 1 --2- 172.21.15.89:0/2711389963 >> v2:172.21.15.172:3300/0 conn(0x7f4078070f20 0x7f40780c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f40777fe700 1 --2- 172.21.15.89:0/2711389963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40780712f0 0x7f407806e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f407cd19700 1 --2- 172.21.15.89:0/2711389963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f407806e790 0x7f407806b190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f407806cd40 tx=0x7f407000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec518f8d0c0c6a4c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f4076ffd700 1 -- 172.21.15.89:0/2711389963 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000e070 con 0x7f407806e790 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f4076ffd700 1 -- 172.21.15.89:0/2711389963 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000b9c0 con 0x7f407806e790 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.564+0000 7f4076ffd700 1 -- 172.21.15.89:0/2711389963 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000bd70 con 0x7f407806e790 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 -- 172.21.15.89:0/2711389963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f407806e790 msgr2=0x7f407806b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 --2- 172.21.15.89:0/2711389963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f407806e790 0x7f407806b190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f407806cd40 tx=0x7f407000b330 comp rx=0 tx=0).stop 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 -- 172.21.15.89:0/2711389963 shutdown_connections 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 --2- 172.21.15.89:0/2711389963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f407806e790 0x7f407806b190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 --2- 172.21.15.89:0/2711389963 >> v2:172.21.15.172:3300/0 conn(0x7f4078070f20 0x7f40780c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 --2- 172.21.15.89:0/2711389963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40780712f0 0x7f407806e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 -- 172.21.15.89:0/2711389963 >> 172.21.15.89:0/2711389963 conn(0x7f40780cf1c0 msgr2=0x7f40780cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 -- 172.21.15.89:0/2711389963 shutdown_connections 2023-07-06T02:23:20.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.565+0000 7f407e77c700 1 -- 172.21.15.89:0/2711389963 wait complete. 2023-07-06T02:23:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.566+0000 7f407e77c700 1 Processor -- start 2023-07-06T02:23:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.566+0000 7f407e77c700 1 -- start start 2023-07-06T02:23:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407e77c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f407806e790 0x7f407815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407e77c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4078070f20 0x7f40780c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407e77c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40780712f0 0x7f40780c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407e77c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780aed30 con 0x7f40780712f0 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407e77c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407815bf10 con 0x7f407806e790 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407e77c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407815a8a0 con 0x7f4078070f20 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40780712f0 0x7f40780c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40780712f0 0x7f40780c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33048/0 (socket says 172.21.15.89:33048) 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 -- 172.21.15.89:0/547424481 learned_addr learned my addr 172.21.15.89:0/547424481 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f40777fe700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4078070f20 0x7f40780c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f4077fff700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f407806e790 0x7f407815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 -- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4078070f20 msgr2=0x7f40780c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4078070f20 0x7f40780c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 -- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f407806e790 msgr2=0x7f407815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f407806e790 0x7f407815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.567+0000 7f407cd19700 1 -- 172.21.15.89:0/547424481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4070009b30 con 0x7f40780712f0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.568+0000 7f407cd19700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40780712f0 0x7f40780c89d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f407000b370 tx=0x7f4070004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.568+0000 7f4074ff9700 1 -- 172.21.15.89:0/547424481 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070004250 con 0x7f40780712f0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.568+0000 7f4074ff9700 1 -- 172.21.15.89:0/547424481 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40700043b0 con 0x7f40780712f0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.568+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407815ab80 con 0x7f40780712f0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.568+0000 7f4074ff9700 1 -- 172.21.15.89:0/547424481 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070004760 con 0x7f40780712f0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.568+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4078152d90 con 0x7f40780712f0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.569+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407815b0a0 con 0x7f40780712f0 2023-07-06T02:23:20.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.569+0000 7f4074ff9700 1 -- 172.21.15.89:0/547424481 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f407001c440 con 0x7f40780712f0 2023-07-06T02:23:20.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.570+0000 7f4074ff9700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.570+0000 7f4074ff9700 1 -- 172.21.15.89:0/547424481 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f407008aca0 con 0x7f40780712f0 2023-07-06T02:23:20.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.570+0000 7f4077fff700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.570+0000 7f4077fff700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f406406ace0 0x7f406406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40680046e0 tx=0x7f4068004040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.574+0000 7f4074ff9700 1 -- 172.21.15.89:0/547424481 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4070055af0 con 0x7f40780712f0 2023-07-06T02:23:20.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.762+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4078074270 con 0x7f40780712f0 2023-07-06T02:23:20.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.763+0000 7f4074ff9700 1 -- 172.21.15.89:0/547424481 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f40700200a0 con 0x7f40780712f0 2023-07-06T02:23:20.766 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:20.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.764+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f406406ace0 msgr2=0x7f406406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.764+0000 7f407e77c700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f406406ace0 0x7f406406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40680046e0 tx=0x7f4068004040 comp rx=0 tx=0).stop 2023-07-06T02:23:20.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.764+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40780712f0 msgr2=0x7f40780c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.764+0000 7f407e77c700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40780712f0 0x7f40780c89d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f407000b370 tx=0x7f4070004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 shutdown_connections 2023-07-06T02:23:20.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40780712f0 0x7f40780c89d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f407806e790 0x7f407815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 --2- 172.21.15.89:0/547424481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4078070f20 0x7f40780c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 >> 172.21.15.89:0/547424481 conn(0x7f40780cf1c0 msgr2=0x7f407815c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:20.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 shutdown_connections 2023-07-06T02:23:20.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.765+0000 7f407e77c700 1 -- 172.21.15.89:0/547424481 wait complete. 2023-07-06T02:23:20.779 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:20.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.896+0000 7fb906492700 1 Processor -- start 2023-07-06T02:23:20.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.896+0000 7fb906492700 1 -- start start 2023-07-06T02:23:20.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.897+0000 7fb906492700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb90015c500 0x7fb90006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.897+0000 7fb906492700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9000cb9f0 con 0x7fb900070f20 2023-07-06T02:23:20.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.897+0000 7fb906492700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90015d8e0 con 0x7fb900160710 2023-07-06T02:23:20.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.897+0000 7fb906492700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90006f310 con 0x7fb90015c500 2023-07-06T02:23:20.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.897+0000 7fb8ff7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb900070f20 0x7fb900160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57954/0 (socket says 172.21.15.89:57954) 2023-07-06T02:23:20.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.897+0000 7fb8ff7fe700 1 -- 172.21.15.89:0/4045270495 learned_addr learned my addr 172.21.15.89:0/4045270495 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:20.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8fffff700 1 --2- 172.21.15.89:0/4045270495 >> v2:172.21.15.172:3300/0 conn(0x7fb90015c500 0x7fb90006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638126246 0 0) 0x7fb9000cb9f0 con 0x7fb900070f20 2023-07-06T02:23:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e80035b0 con 0x7fb900070f20 2023-07-06T02:23:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137995394 0 0) 0x7fb8e80035b0 con 0x7fb900070f20 2023-07-06T02:23:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 >> v2:172.21.15.172:3300/0 conn(0x7fb90015c500 msgr2=0x7fb90006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 --2- 172.21.15.89:0/4045270495 >> v2:172.21.15.172:3300/0 conn(0x7fb90015c500 0x7fb90006edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 >> v1:172.21.15.172:6789/0 conn(0x7fb900160710 legacy=0x7fb9000ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb900160be0 con 0x7fb900070f20 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fb8f0001670 con 0x7fb900070f20 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 >> v1:172.21.15.89:6789/0 conn(0x7fb900070f20 legacy=0x7fb900160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8003a00 0x7fb8e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8e8004850 0x7fb8e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e8005200 0x7fb8e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90006f310 con 0x7fb8e8005200 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800c080 con 0x7fb8e8003a00 2023-07-06T02:23:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800c240 con 0x7fb8e8004850 2023-07-06T02:23:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb904a2f700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e8005200 0x7fb8e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8fffff700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8003a00 0x7fb8e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb8ff7fe700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8e8004850 0x7fb8e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb904a2f700 1 -- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8e8004850 msgr2=0x7fb8e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb904a2f700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8e8004850 0x7fb8e8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb904a2f700 1 -- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8003a00 msgr2=0x7fb8e8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb904a2f700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8003a00 0x7fb8e8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb904a2f700 1 -- 172.21.15.89:0/4045270495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e80035b0 con 0x7fb8e8005200 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.898+0000 7fb904a2f700 1 -- 172.21.15.89:0/4045270495 reap_dead start 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb904a2f700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e8005200 0x7fb8e800ba10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c160 tx=0x7fb8e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2784d13ce3775e78 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8001db0 con 0x7fb8e8005200 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800cca0 con 0x7fb8e8005200 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb8feffd700 1 -- 172.21.15.89:0/4045270495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8003c40 con 0x7fb8e8005200 2023-07-06T02:23:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb906492700 1 -- 172.21.15.89:0/4045270495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e8005200 msgr2=0x7fb8e800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb906492700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e8005200 0x7fb8e800ba10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c160 tx=0x7fb8e8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb906492700 1 -- 172.21.15.89:0/4045270495 shutdown_connections 2023-07-06T02:23:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb906492700 1 --2- 172.21.15.89:0/4045270495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8e8005200 0x7fb8e800ba10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb906492700 1 -- 172.21.15.89:0/4045270495 >> 172.21.15.89:0/4045270495 conn(0x7fb9000cf1c0 msgr2=0x7fb9000cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb906492700 1 -- 172.21.15.89:0/4045270495 shutdown_connections 2023-07-06T02:23:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.899+0000 7fb906492700 1 -- 172.21.15.89:0/4045270495 wait complete. 2023-07-06T02:23:20.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.900+0000 7fb906492700 1 Processor -- start 2023-07-06T02:23:20.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb906492700 1 -- start start 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb906492700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8005200 0x7fb9000b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb906492700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9000ac940 0x7fb9000ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb906492700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9000afb30 0x7fb9001f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb906492700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000ad9c0 con 0x7fb9000ac940 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb906492700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000ae2e0 con 0x7fb8e8005200 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb906492700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000b4b00 con 0x7fb9000afb30 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.901+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9000ac940 0x7fb9000ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9000ac940 0x7fb9000ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33066/0 (socket says 172.21.15.89:33066) 2023-07-06T02:23:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 -- 172.21.15.89:0/969157863 learned_addr learned my addr 172.21.15.89:0/969157863 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb904a2f700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9000afb30 0x7fb9001f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8fffff700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8005200 0x7fb9000b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 -- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9000afb30 msgr2=0x7fb9001f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9000afb30 0x7fb9001f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 -- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8005200 msgr2=0x7fb9000b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8005200 0x7fb9000b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 -- 172.21.15.89:0/969157863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9000ace40 con 0x7fb9000ac940 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8ff7fe700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9000ac940 0x7fb9000ce160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0007290 tx=0x7fb8f00075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8fcff9700 1 -- 172.21.15.89:0/969157863 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0012560 con 0x7fb9000ac940 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8fcff9700 1 -- 172.21.15.89:0/969157863 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f00126c0 con 0x7fb9000ac940 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9000ad120 con 0x7fb9000ac940 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb8fcff9700 1 -- 172.21.15.89:0/969157863 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0012a50 con 0x7fb9000ac940 2023-07-06T02:23:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.902+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9000ad6e0 con 0x7fb9000ac940 2023-07-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.903+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb900074270 con 0x7fb9000ac940 2023-07-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.904+0000 7fb8fcff9700 1 -- 172.21.15.89:0/969157863 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8f0012bb0 con 0x7fb9000ac940 2023-07-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.904+0000 7fb8fcff9700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb8ec06ace0 0x7fb8ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.904+0000 7fb8fcff9700 1 -- 172.21.15.89:0/969157863 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb8f0089590 con 0x7fb9000ac940 2023-07-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.904+0000 7fb8fffff700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb8ec06ace0 0x7fb8ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.904+0000 7fb8fffff700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb8ec06ace0 0x7fb8ec06d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009be0 tx=0x7fb8f4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:20.908+0000 7fb8fcff9700 1 -- 172.21.15.89:0/969157863 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8f0054350 con 0x7fb9000ac940 2023-07-06T02:23:21.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.140+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb900071dd0 con 0x7fb9000ac940 2023-07-06T02:23:21.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.140+0000 7fb8fcff9700 1 -- 172.21.15.89:0/969157863 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fb8f0057970 con 0x7fb9000ac940 2023-07-06T02:23:21.144 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:21.144 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb8ec06ace0 msgr2=0x7fb8ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb8ec06ace0 0x7fb8ec06d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009be0 tx=0x7fb8f4009210 comp rx=0 tx=0).stop 2023-07-06T02:23:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9000ac940 msgr2=0x7fb9000ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9000ac940 0x7fb9000ce160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0007290 tx=0x7fb8f00075a0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 shutdown_connections 2023-07-06T02:23:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9000afb30 0x7fb9001f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8005200 0x7fb9000b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb8ec06ace0 0x7fb8ec06d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 --2- 172.21.15.89:0/969157863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9000ac940 0x7fb9000ce160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.142+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 >> 172.21.15.89:0/969157863 conn(0x7fb9000cf1c0 msgr2=0x7fb90015f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.143+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 shutdown_connections 2023-07-06T02:23:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.143+0000 7fb906492700 1 -- 172.21.15.89:0/969157863 wait complete. 2023-07-06T02:23:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:21.158 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.276+0000 7f703740a700 1 Processor -- start 2023-07-06T02:23:21.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.277+0000 7f703740a700 1 -- start start 2023-07-06T02:23:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.277+0000 7f703740a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f703006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.277+0000 7f703740a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f703006e790 0x7f703006b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.277+0000 7f703740a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703006f290 con 0x7f7030070f20 2023-07-06T02:23:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.277+0000 7f703740a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703006ecd0 con 0x7f703006e790 2023-07-06T02:23:21.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.277+0000 7f703740a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703006efb0 con 0x7f70300712f0 2023-07-06T02:23:21.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70359a7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f703006e790 0x7f703006b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70349a5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f703006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70359a7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f703006e790 0x7f703006b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:60066/0 (socket says 172.21.15.89:60066) 2023-07-06T02:23:21.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70351a6700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7030070f20 0x7f70300c9a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33730/0 (socket says 172.21.15.89:33730) 2023-07-06T02:23:21.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70349a5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f703006e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:56282/0 (socket says 172.21.15.89:56282) 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70351a6700 1 -- 172.21.15.89:0/2376048864 learned_addr learned my addr 172.21.15.89:0/2376048864 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70359a7700 1 -- 172.21.15.89:0/2376048864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 msgr2=0x7f703006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70359a7700 1 --2- 172.21.15.89:0/2376048864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f703006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70359a7700 1 -- 172.21.15.89:0/2376048864 >> v1:172.21.15.172:6789/0 conn(0x7f7030070f20 legacy=0x7f70300c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70359a7700 1 -- 172.21.15.89:0/2376048864 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703006a990 con 0x7f703006e790 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f70359a7700 1 --2- 172.21.15.89:0/2376048864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703006e790 0x7f703006b190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f702800bfa0 tx=0x7f702800d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98eb7f0ab644a31 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f7027fff700 1 -- 172.21.15.89:0/2376048864 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028010070 con 0x7f703006e790 2023-07-06T02:23:21.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.278+0000 7f7027fff700 1 -- 172.21.15.89:0/2376048864 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800da00 con 0x7f703006e790 2023-07-06T02:23:21.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f7027fff700 1 -- 172.21.15.89:0/2376048864 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800dd90 con 0x7f703006e790 2023-07-06T02:23:21.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 -- 172.21.15.89:0/2376048864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703006e790 msgr2=0x7f703006b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 --2- 172.21.15.89:0/2376048864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703006e790 0x7f703006b190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f702800bfa0 tx=0x7f702800d330 comp rx=0 tx=0).stop 2023-07-06T02:23:21.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 -- 172.21.15.89:0/2376048864 shutdown_connections 2023-07-06T02:23:21.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 --2- 172.21.15.89:0/2376048864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703006e790 0x7f703006b190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 --2- 172.21.15.89:0/2376048864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f703006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 -- 172.21.15.89:0/2376048864 >> 172.21.15.89:0/2376048864 conn(0x7f70300cf1c0 msgr2=0x7f70300cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:21.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 -- 172.21.15.89:0/2376048864 shutdown_connections 2023-07-06T02:23:21.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.279+0000 7f703740a700 1 -- 172.21.15.89:0/2376048864 wait complete. 2023-07-06T02:23:21.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.280+0000 7f703740a700 1 Processor -- start 2023-07-06T02:23:21.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.280+0000 7f703740a700 1 -- start start 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f703740a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f703006e790 0x7f70300b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f703740a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7030070f20 0x7f70301f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f703740a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f70300ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f703740a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70300b0b60 con 0x7f703006e790 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f703740a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70300b14c0 con 0x7f7030070f20 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f703740a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70301f6740 con 0x7f70300712f0 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f703006e790 0x7f70300b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f703006e790 0x7f70300b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33078/0 (socket says 172.21.15.89:33078) 2023-07-06T02:23:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 -- 172.21.15.89:0/2658787525 learned_addr learned my addr 172.21.15.89:0/2658787525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70349a5700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7030070f20 0x7f70301f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70359a7700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f70300ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 -- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 msgr2=0x7f70300ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f70300ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 -- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7030070f20 msgr2=0x7f70301f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7030070f20 0x7f70301f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.281+0000 7f70351a6700 1 -- 172.21.15.89:0/2658787525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702800bc80 con 0x7f703006e790 2023-07-06T02:23:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.282+0000 7f70351a6700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f703006e790 0x7f70300b0f80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f702000c490 tx=0x7f702000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.282+0000 7f7025ffb700 1 -- 172.21.15.89:0/2658787525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020005d70 con 0x7f703006e790 2023-07-06T02:23:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.282+0000 7f7025ffb700 1 -- 172.21.15.89:0/2658787525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70200070d0 con 0x7f703006e790 2023-07-06T02:23:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.282+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70300afee0 con 0x7f703006e790 2023-07-06T02:23:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.282+0000 7f7025ffb700 1 -- 172.21.15.89:0/2658787525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020007460 con 0x7f703006e790 2023-07-06T02:23:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.282+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70300b04a0 con 0x7f703006e790 2023-07-06T02:23:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.283+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7030074270 con 0x7f703006e790 2023-07-06T02:23:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.283+0000 7f7025ffb700 1 -- 172.21.15.89:0/2658787525 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70200075c0 con 0x7f703006e790 2023-07-06T02:23:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.284+0000 7f7025ffb700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f701406ace0 0x7f701406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.284+0000 7f7025ffb700 1 -- 172.21.15.89:0/2658787525 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7020088c10 con 0x7f703006e790 2023-07-06T02:23:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.284+0000 7f70349a5700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f701406ace0 0x7f701406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.284+0000 7f70349a5700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f701406ace0 0x7f701406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70300b21c0 tx=0x7f702c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.288+0000 7f7025ffb700 1 -- 172.21.15.89:0/2658787525 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7020053a60 con 0x7f703006e790 2023-07-06T02:23:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.482+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70300c9580 con 0x7f703006e790 2023-07-06T02:23:21.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.483+0000 7f7025ffb700 1 -- 172.21.15.89:0/2658787525 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f7020017090 con 0x7f703006e790 2023-07-06T02:23:21.486 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:21.487 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:21.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f701406ace0 msgr2=0x7f701406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f701406ace0 0x7f701406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70300b21c0 tx=0x7f702c008040 comp rx=0 tx=0).stop 2023-07-06T02:23:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f703006e790 msgr2=0x7f70300b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f703006e790 0x7f70300b0f80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f702000c490 tx=0x7f702000c850 comp rx=0 tx=0).stop 2023-07-06T02:23:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 shutdown_connections 2023-07-06T02:23:21.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70300712f0 0x7f70300ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f701406ace0 0x7f701406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f703006e790 0x7f70300b0f80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.485+0000 7f703740a700 1 --2- 172.21.15.89:0/2658787525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7030070f20 0x7f70301f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.486+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 >> 172.21.15.89:0/2658787525 conn(0x7f70300cf1c0 msgr2=0x7f70300b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.486+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 shutdown_connections 2023-07-06T02:23:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.486+0000 7f703740a700 1 -- 172.21.15.89:0/2658787525 wait complete. 2023-07-06T02:23:21.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:21.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.623+0000 7f2382262700 1 Processor -- start 2023-07-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.623+0000 7f2382262700 1 -- start start 2023-07-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f2382262700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f237c156ab0 0x7f237c160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f2382262700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f237c1600b0 0x7f237c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f2382262700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237c0cb3d0 con 0x7f237c160bb0 2023-07-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f2382262700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237c156e80 con 0x7f237c1600b0 2023-07-06T02:23:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f2382262700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237c0ca370 con 0x7f237c156ab0 2023-07-06T02:23:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f237bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f237c1600b0 0x7f237c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f237bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f237c1600b0 0x7f237c0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33086/0 (socket says 172.21.15.89:33086) 2023-07-06T02:23:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f237affd700 1 --1- 172.21.15.89:0/3245275361 >> v1:172.21.15.89:6789/0 conn(0x7f237c160bb0 0x7f237c0b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57988/0 (socket says 172.21.15.89:57988) 2023-07-06T02:23:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.624+0000 7f237bfff700 1 -- 172.21.15.89:0/3245275361 learned_addr learned my addr 172.21.15.89:0/3245275361 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237b7fe700 1 --2- 172.21.15.89:0/3245275361 >> v2:172.21.15.172:3300/0 conn(0x7f237c156ab0 0x7f237c160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237bfff700 1 -- 172.21.15.89:0/3245275361 >> v2:172.21.15.172:3300/0 conn(0x7f237c156ab0 msgr2=0x7f237c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237bfff700 1 --2- 172.21.15.89:0/3245275361 >> v2:172.21.15.172:3300/0 conn(0x7f237c156ab0 0x7f237c160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237bfff700 1 -- 172.21.15.89:0/3245275361 >> v1:172.21.15.89:6789/0 conn(0x7f237c160bb0 legacy=0x7f237c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237bfff700 1 -- 172.21.15.89:0/3245275361 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c0ca610 con 0x7f237c1600b0 2023-07-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237a7fc700 1 -- 172.21.15.89:0/3245275361 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910004731 0 0) 0x7f237c0cb3d0 con 0x7f237c160bb0 2023-07-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237bfff700 1 --2- 172.21.15.89:0/3245275361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c1600b0 0x7f237c0cd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f237c06e9e0 tx=0x7f237400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6a5fb04fb85087 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237a7fc700 1 -- 172.21.15.89:0/3245275361 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237400e070 con 0x7f237c1600b0 2023-07-06T02:23:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237a7fc700 1 -- 172.21.15.89:0/3245275361 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237400b9c0 con 0x7f237c1600b0 2023-07-06T02:23:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.625+0000 7f237a7fc700 1 -- 172.21.15.89:0/3245275361 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237400bd70 con 0x7f237c1600b0 2023-07-06T02:23:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 -- 172.21.15.89:0/3245275361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c1600b0 msgr2=0x7f237c0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 --2- 172.21.15.89:0/3245275361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c1600b0 0x7f237c0cd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f237c06e9e0 tx=0x7f237400b330 comp rx=0 tx=0).stop 2023-07-06T02:23:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 -- 172.21.15.89:0/3245275361 shutdown_connections 2023-07-06T02:23:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 --2- 172.21.15.89:0/3245275361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c1600b0 0x7f237c0cd700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 --2- 172.21.15.89:0/3245275361 >> v2:172.21.15.172:3300/0 conn(0x7f237c156ab0 0x7f237c160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 -- 172.21.15.89:0/3245275361 >> 172.21.15.89:0/3245275361 conn(0x7f237c0cc490 msgr2=0x7f237c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 -- 172.21.15.89:0/3245275361 shutdown_connections 2023-07-06T02:23:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.626+0000 7f2382262700 1 -- 172.21.15.89:0/3245275361 wait complete. 2023-07-06T02:23:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.627+0000 7f2382262700 1 Processor -- start 2023-07-06T02:23:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.627+0000 7f2382262700 1 -- start start 2023-07-06T02:23:21.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f2382262700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c156ab0 0x7f237c0abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f2382262700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f237c1600b0 0x7f237c144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f2382262700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237c1e5dd0 0x7f237c1e6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f2382262700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1428b0 con 0x7f237c156ab0 2023-07-06T02:23:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f2382262700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c142b90 con 0x7f237c1600b0 2023-07-06T02:23:21.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f2382262700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c142e70 con 0x7f237c1e5dd0 2023-07-06T02:23:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f237b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c156ab0 0x7f237c0abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.628+0000 7f237b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c156ab0 0x7f237c0abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33090/0 (socket says 172.21.15.89:33090) 2023-07-06T02:23:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237b7fe700 1 -- 172.21.15.89:0/3110306278 learned_addr learned my addr 172.21.15.89:0/3110306278 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237affd700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f237c1600b0 0x7f237c144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237bfff700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237c1e5dd0 0x7f237c1e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237b7fe700 1 -- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237c1e5dd0 msgr2=0x7f237c1e6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237b7fe700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237c1e5dd0 0x7f237c1e6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237b7fe700 1 -- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f237c1600b0 msgr2=0x7f237c144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237b7fe700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f237c1600b0 0x7f237c144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237b7fe700 1 -- 172.21.15.89:0/3110306278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374009b30 con 0x7f237c156ab0 2023-07-06T02:23:21.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f237b7fe700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c156ab0 0x7f237c0abda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f236400d800 tx=0x7f236400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f2373fff700 1 -- 172.21.15.89:0/3110306278 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640125f0 con 0x7f237c156ab0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f2373fff700 1 -- 172.21.15.89:0/3110306278 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364012750 con 0x7f237c156ab0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c143150 con 0x7f237c156ab0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f2373fff700 1 -- 172.21.15.89:0/3110306278 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364012b00 con 0x7f237c156ab0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.629+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c143710 con 0x7f237c156ab0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.630+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c1438d0 con 0x7f237c156ab0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.631+0000 7f2373fff700 1 -- 172.21.15.89:0/3110306278 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2364012c60 con 0x7f237c156ab0 2023-07-06T02:23:21.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.631+0000 7f2373fff700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f235c06ace0 0x7f235c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.631+0000 7f2373fff700 1 -- 172.21.15.89:0/3110306278 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2364089b20 con 0x7f237c156ab0 2023-07-06T02:23:21.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.631+0000 7f237affd700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f235c06ace0 0x7f235c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.631+0000 7f237affd700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f235c06ace0 0x7f235c06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f237c0b5420 tx=0x7f236c007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.635+0000 7f2373fff700 1 -- 172.21.15.89:0/3110306278 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23640548e0 con 0x7f237c156ab0 2023-07-06T02:23:21.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.820+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f237c143a80 con 0x7f237c156ab0 2023-07-06T02:23:21.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.821+0000 7f2373fff700 1 -- 172.21.15.89:0/3110306278 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2364057f00 con 0x7f237c156ab0 2023-07-06T02:23:21.824 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:21.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f235c06ace0 msgr2=0x7f235c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f235c06ace0 0x7f235c06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f237c0b5420 tx=0x7f236c007be0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c156ab0 msgr2=0x7f237c0abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c156ab0 0x7f237c0abda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f236400d800 tx=0x7f236400dbc0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 shutdown_connections 2023-07-06T02:23:21.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237c1e5dd0 0x7f237c1e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f235c06ace0 0x7f235c06d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f237c156ab0 0x7f237c0abda0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 --2- 172.21.15.89:0/3110306278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f237c1600b0 0x7f237c144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 >> 172.21.15.89:0/3110306278 conn(0x7f237c0cc490 msgr2=0x7f237c15ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:21.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 shutdown_connections 2023-07-06T02:23:21.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.823+0000 7f2382262700 1 -- 172.21.15.89:0/3110306278 wait complete. 2023-07-06T02:23:21.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:21.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.965+0000 7f14afe09700 1 Processor -- start 2023-07-06T02:23:21.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.965+0000 7f14afe09700 1 -- start start 2023-07-06T02:23:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14afe09700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14a8070970 0x7f14a8157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14afe09700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a80cd350 con 0x7f14a806dc10 2023-07-06T02:23:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14afe09700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a806e110 con 0x7f14a80705a0 2023-07-06T02:23:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14afe09700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a806e3f0 con 0x7f14a8070970 2023-07-06T02:23:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14ae3a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14a8070970 0x7f14a8157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14ae3a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14a8070970 0x7f14a8157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33102/0 (socket says 172.21.15.89:33102) 2023-07-06T02:23:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14ae3a6700 1 -- 172.21.15.89:0/2784450109 learned_addr learned my addr 172.21.15.89:0/2784450109 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.966+0000 7f14ad3a4700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f14a806dc10 0x7f14a815a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58002/0 (socket says 172.21.15.89:58002) 2023-07-06T02:23:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14ae3a6700 1 -- 172.21.15.89:0/2784450109 >> v1:172.21.15.172:6789/0 conn(0x7f14a80705a0 legacy=0x7f14a80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14ae3a6700 1 -- 172.21.15.89:0/2784450109 >> v1:172.21.15.89:6789/0 conn(0x7f14a806dc10 legacy=0x7f14a815a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14ae3a6700 1 -- 172.21.15.89:0/2784450109 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a806e6d0 con 0x7f14a8070970 2023-07-06T02:23:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14acba3700 1 -- 172.21.15.89:0/2784450109 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850322167 0 0) 0x7f14a80cd350 con 0x7f14a806dc10 2023-07-06T02:23:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14ae3a6700 1 --2- 172.21.15.89:0/2784450109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8070970 0x7f14a8157950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009eb0 tx=0x7f14a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adec5f943b443d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14acba3700 1 -- 172.21.15.89:0/2784450109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000e070 con 0x7f14a8070970 2023-07-06T02:23:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14acba3700 1 -- 172.21.15.89:0/2784450109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a0003a20 con 0x7f14a8070970 2023-07-06T02:23:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14acba3700 1 -- 172.21.15.89:0/2784450109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003dd0 con 0x7f14a8070970 2023-07-06T02:23:21.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14afe09700 1 -- 172.21.15.89:0/2784450109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8070970 msgr2=0x7f14a8157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14afe09700 1 --2- 172.21.15.89:0/2784450109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8070970 0x7f14a8157950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009eb0 tx=0x7f14a000b330 comp rx=0 tx=0).stop 2023-07-06T02:23:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14afe09700 1 -- 172.21.15.89:0/2784450109 shutdown_connections 2023-07-06T02:23:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.967+0000 7f14afe09700 1 --2- 172.21.15.89:0/2784450109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a8070970 0x7f14a8157950 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.968+0000 7f14afe09700 1 -- 172.21.15.89:0/2784450109 >> 172.21.15.89:0/2784450109 conn(0x7f14a80d0b00 msgr2=0x7f14a80d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.968+0000 7f14afe09700 1 -- 172.21.15.89:0/2784450109 shutdown_connections 2023-07-06T02:23:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.968+0000 7f14afe09700 1 -- 172.21.15.89:0/2784450109 wait complete. 2023-07-06T02:23:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.969+0000 7f14afe09700 1 Processor -- start 2023-07-06T02:23:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.969+0000 7f14afe09700 1 -- start start 2023-07-06T02:23:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.969+0000 7f14afe09700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14a806dc10 0x7f14a8159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14afe09700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a80705a0 0x7f14a8159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14afe09700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14a8070970 0x7f14a81f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14afe09700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80c73d0 con 0x7f14a80705a0 2023-07-06T02:23:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14afe09700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a81f2d50 con 0x7f14a806dc10 2023-07-06T02:23:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14afe09700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a81f3030 con 0x7f14a8070970 2023-07-06T02:23:21.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a80705a0 0x7f14a8159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a80705a0 0x7f14a8159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33106/0 (socket says 172.21.15.89:33106) 2023-07-06T02:23:21.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 -- 172.21.15.89:0/3082244300 learned_addr learned my addr 172.21.15.89:0/3082244300 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:21.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14adba5700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14a806dc10 0x7f14a8159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ae3a6700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14a8070970 0x7f14a81f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 -- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14a8070970 msgr2=0x7f14a81f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14a8070970 0x7f14a81f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 -- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14a806dc10 msgr2=0x7f14a8159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:21.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14a806dc10 0x7f14a8159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:21.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 -- 172.21.15.89:0/3082244300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a0009b30 con 0x7f14a80705a0 2023-07-06T02:23:21.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14ad3a4700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a80705a0 0x7f14a8159da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c520 tx=0x7f14a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f149e7fc700 1 -- 172.21.15.89:0/3082244300 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004c80 con 0x7f14a80705a0 2023-07-06T02:23:21.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f149e7fc700 1 -- 172.21.15.89:0/3082244300 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4004e00 con 0x7f14a80705a0 2023-07-06T02:23:21.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a80c7c70 con 0x7f14a80705a0 2023-07-06T02:23:21.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f149e7fc700 1 -- 172.21.15.89:0/3082244300 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4007310 con 0x7f14a80705a0 2023-07-06T02:23:21.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.970+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a80c8230 con 0x7f14a80705a0 2023-07-06T02:23:21.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.971+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a80c8420 con 0x7f14a80705a0 2023-07-06T02:23:21.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.972+0000 7f149e7fc700 1 -- 172.21.15.89:0/3082244300 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14a4007470 con 0x7f14a80705a0 2023-07-06T02:23:21.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.972+0000 7f149e7fc700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f148c06ace0 0x7f148c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:21.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.972+0000 7f149e7fc700 1 -- 172.21.15.89:0/3082244300 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f14a4089400 con 0x7f14a80705a0 2023-07-06T02:23:21.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.972+0000 7f14adba5700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f148c06ace0 0x7f148c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:21.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.973+0000 7f14adba5700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f148c06ace0 0x7f148c06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1498007fd0 tx=0x7f1498007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:21.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:21.977+0000 7f149e7fc700 1 -- 172.21.15.89:0/3082244300 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f14a4054120 con 0x7f14a80705a0 2023-07-06T02:23:22.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.162+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14a8074270 con 0x7f14a80705a0 2023-07-06T02:23:22.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.163+0000 7f149e7fc700 1 -- 172.21.15.89:0/3082244300 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14a4057740 con 0x7f14a80705a0 2023-07-06T02:23:22.167 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f148c06ace0 msgr2=0x7f148c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f148c06ace0 0x7f148c06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1498007fd0 tx=0x7f1498007ec0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a80705a0 msgr2=0x7f14a8159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a80705a0 0x7f14a8159da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c520 tx=0x7f14a400c830 comp rx=0 tx=0).stop 2023-07-06T02:23:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 shutdown_connections 2023-07-06T02:23:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14a8070970 0x7f14a81f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14a806dc10 0x7f14a8159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f148c06ace0 0x7f148c06d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 --2- 172.21.15.89:0/3082244300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14a80705a0 0x7f14a8159da0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 >> 172.21.15.89:0/3082244300 conn(0x7f14a80d0b00 msgr2=0x7f14a815d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 shutdown_connections 2023-07-06T02:23:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.165+0000 7f14afe09700 1 -- 172.21.15.89:0/3082244300 wait complete. 2023-07-06T02:23:22.180 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:22.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.300+0000 7f7a8c857700 1 Processor -- start 2023-07-06T02:23:22.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.300+0000 7f7a8c857700 1 -- start start 2023-07-06T02:23:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a8c857700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a880b1490 0x7f7a880b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a8c857700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a8806bfa0 0x7f7a8806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a8c857700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a880cab50 con 0x7f7a8815c690 2023-07-06T02:23:22.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a8c857700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8815de50 con 0x7f7a880b1490 2023-07-06T02:23:22.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a8c857700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880b1e50 con 0x7f7a8806bfa0 2023-07-06T02:23:22.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a85d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a880b1490 0x7f7a880b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a85d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a880b1490 0x7f7a880b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33114/0 (socket says 172.21.15.89:33114) 2023-07-06T02:23:22.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a8659c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7a8815c690 0x7f7a880cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58014/0 (socket says 172.21.15.89:58014) 2023-07-06T02:23:22.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.301+0000 7f7a85d9b700 1 -- 172.21.15.89:0/4224700477 learned_addr learned my addr 172.21.15.89:0/4224700477 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:22.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a86d9d700 1 --2- 172.21.15.89:0/4224700477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a8806bfa0 0x7f7a8806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a85d9b700 1 -- 172.21.15.89:0/4224700477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a8806bfa0 msgr2=0x7f7a8806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:22.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a85d9b700 1 --2- 172.21.15.89:0/4224700477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a8806bfa0 0x7f7a8806c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a85d9b700 1 -- 172.21.15.89:0/4224700477 >> v1:172.21.15.89:6789/0 conn(0x7f7a8815c690 legacy=0x7f7a880cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a8559a700 1 -- 172.21.15.89:0/4224700477 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534812752 0 0) 0x7f7a880cab50 con 0x7f7a8815c690 2023-07-06T02:23:22.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a85d9b700 1 -- 172.21.15.89:0/4224700477 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8806c920 con 0x7f7a880b1490 2023-07-06T02:23:22.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a85d9b700 1 --2- 172.21.15.89:0/4224700477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a880b1490 0x7f7a880b1910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098f0 tx=0x7f7a78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46a5c5b94044b33 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a8559a700 1 -- 172.21.15.89:0/4224700477 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800b8a0 con 0x7f7a880b1490 2023-07-06T02:23:22.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a8559a700 1 -- 172.21.15.89:0/4224700477 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7800ba00 con 0x7f7a880b1490 2023-07-06T02:23:22.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a8559a700 1 -- 172.21.15.89:0/4224700477 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800bd90 con 0x7f7a880b1490 2023-07-06T02:23:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a8c857700 1 -- 172.21.15.89:0/4224700477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a880b1490 msgr2=0x7f7a880b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.302+0000 7f7a8c857700 1 --2- 172.21.15.89:0/4224700477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a880b1490 0x7f7a880b1910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098f0 tx=0x7f7a78009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:22.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.303+0000 7f7a8c857700 1 -- 172.21.15.89:0/4224700477 shutdown_connections 2023-07-06T02:23:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.303+0000 7f7a8c857700 1 --2- 172.21.15.89:0/4224700477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a8806bfa0 0x7f7a8806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.303+0000 7f7a8c857700 1 --2- 172.21.15.89:0/4224700477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a880b1490 0x7f7a880b1910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.303+0000 7f7a8c857700 1 -- 172.21.15.89:0/4224700477 >> 172.21.15.89:0/4224700477 conn(0x7f7a880cc490 msgr2=0x7f7a880cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.303+0000 7f7a8c857700 1 -- 172.21.15.89:0/4224700477 shutdown_connections 2023-07-06T02:23:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.303+0000 7f7a8c857700 1 -- 172.21.15.89:0/4224700477 wait complete. 2023-07-06T02:23:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.304+0000 7f7a8c857700 1 Processor -- start 2023-07-06T02:23:22.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.304+0000 7f7a8c857700 1 -- start start 2023-07-06T02:23:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8c857700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a8806bfa0 0x7f7a88154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8c857700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a880b1490 0x7f7a880b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8c857700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a8815c690 0x7f7a88158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8c857700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a88154e40 con 0x7f7a8806bfa0 2023-07-06T02:23:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8c857700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a88155820 con 0x7f7a8815c690 2023-07-06T02:23:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8c857700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880c7870 con 0x7f7a880b1490 2023-07-06T02:23:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a8806bfa0 0x7f7a88154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a8806bfa0 0x7f7a88154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33118/0 (socket says 172.21.15.89:33118) 2023-07-06T02:23:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 -- 172.21.15.89:0/3890936100 learned_addr learned my addr 172.21.15.89:0/3890936100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a86d9d700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a8815c690 0x7f7a88158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a85d9b700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a880b1490 0x7f7a880b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 -- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a880b1490 msgr2=0x7f7a880b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a880b1490 0x7f7a880b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 -- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a8815c690 msgr2=0x7f7a88158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a8815c690 0x7f7a88158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 -- 172.21.15.89:0/3890936100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78009550 con 0x7f7a8806bfa0 2023-07-06T02:23:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a86d9d700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a8815c690 0x7f7a88158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8659c700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a8806bfa0 0x7f7a88154900 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0011e0 tx=0x7f7a7c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a76ffd700 1 -- 172.21.15.89:0/3890936100 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c004c80 con 0x7f7a8806bfa0 2023-07-06T02:23:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a76ffd700 1 -- 172.21.15.89:0/3890936100 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c004e00 con 0x7f7a8806bfa0 2023-07-06T02:23:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a880c7b50 con 0x7f7a8806bfa0 2023-07-06T02:23:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.305+0000 7f7a76ffd700 1 -- 172.21.15.89:0/3890936100 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c007310 con 0x7f7a8806bfa0 2023-07-06T02:23:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.306+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a880c8110 con 0x7f7a8806bfa0 2023-07-06T02:23:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.306+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a88074270 con 0x7f7a8806bfa0 2023-07-06T02:23:22.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.307+0000 7f7a76ffd700 1 -- 172.21.15.89:0/3890936100 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a7c007470 con 0x7f7a8806bfa0 2023-07-06T02:23:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.307+0000 7f7a76ffd700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7a6806ace0 0x7f7a6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.307+0000 7f7a76ffd700 1 -- 172.21.15.89:0/3890936100 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a7c0894d0 con 0x7f7a8806bfa0 2023-07-06T02:23:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.307+0000 7f7a85d9b700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7a6806ace0 0x7f7a6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.308+0000 7f7a85d9b700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7a6806ace0 0x7f7a6806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a78008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.312+0000 7f7a76ffd700 1 -- 172.21.15.89:0/3890936100 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7a7c054320 con 0x7f7a8806bfa0 2023-07-06T02:23:22.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.502+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a880ce640 con 0x7f7a8806bfa0 2023-07-06T02:23:22.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.502+0000 7f7a76ffd700 1 -- 172.21.15.89:0/3890936100 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a7c057940 con 0x7f7a8806bfa0 2023-07-06T02:23:22.505 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:22.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.504+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7a6806ace0 msgr2=0x7f7a6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.504+0000 7f7a8c857700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7a6806ace0 0x7f7a6806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a78008030 comp rx=0 tx=0).stop 2023-07-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a8806bfa0 msgr2=0x7f7a88154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a8806bfa0 0x7f7a88154900 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0011e0 tx=0x7f7a7c00cae0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 shutdown_connections 2023-07-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a8815c690 0x7f7a88158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7a6806ace0 0x7f7a6806d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a8806bfa0 0x7f7a88154900 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 --2- 172.21.15.89:0/3890936100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a880b1490 0x7f7a880b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 >> 172.21.15.89:0/3890936100 conn(0x7f7a880cc490 msgr2=0x7f7a880d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 shutdown_connections 2023-07-06T02:23:22.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.505+0000 7f7a8c857700 1 -- 172.21.15.89:0/3890936100 wait complete. 2023-07-06T02:23:22.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.640+0000 7f5f1844c700 1 Processor -- start 2023-07-06T02:23:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.640+0000 7f5f1844c700 1 -- start start 2023-07-06T02:23:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f1844c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5f1006dc10 0x7f5f1015a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f1844c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f100cd350 con 0x7f5f1015adb0 2023-07-06T02:23:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f1844c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1006e110 con 0x7f5f100705a0 2023-07-06T02:23:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f1844c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1006e3f0 con 0x7f5f1006dc10 2023-07-06T02:23:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f169e9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5f1015adb0 0x7f5f10157950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58032/0 (socket says 172.21.15.89:58032) 2023-07-06T02:23:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f169e9700 1 -- 172.21.15.89:0/2742942710 learned_addr learned my addr 172.21.15.89:0/2742942710 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f159e7700 1 --2- 172.21.15.89:0/2742942710 >> v2:172.21.15.172:3300/0 conn(0x7f5f1006dc10 0x7f5f1015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663960508 0 0) 0x7f5f100cd350 con 0x7f5f1015adb0 2023-07-06T02:23:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.641+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5efc0035b0 con 0x7f5f1015adb0 2023-07-06T02:23:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3032214531 0 0) 0x7f5efc0035b0 con 0x7f5f1015adb0 2023-07-06T02:23:22.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 >> v2:172.21.15.172:3300/0 conn(0x7f5f1006dc10 msgr2=0x7f5f1015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 --2- 172.21.15.89:0/2742942710 >> v2:172.21.15.172:3300/0 conn(0x7f5f1006dc10 0x7f5f1015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 >> v1:172.21.15.172:6789/0 conn(0x7f5f100705a0 legacy=0x7f5f100ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1006e6d0 con 0x7f5f1015adb0 2023-07-06T02:23:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f5f080019f0 con 0x7f5f1015adb0 2023-07-06T02:23:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 >> v1:172.21.15.89:6789/0 conn(0x7f5f1015adb0 legacy=0x7f5f10157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc003a30 0x7f5efc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004850 0x7f5efc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc005200 0x7f5efc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1006e3f0 con 0x7f5efc003a30 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc00c0e0 con 0x7f5efc005200 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc00c270 con 0x7f5efc004850 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f161e8700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc003a30 0x7f5efc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f159e7700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004850 0x7f5efc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f169e9700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc005200 0x7f5efc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f161e8700 1 -- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004850 msgr2=0x7f5efc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f161e8700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004850 0x7f5efc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f161e8700 1 -- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc005200 msgr2=0x7f5efc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f161e8700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc005200 0x7f5efc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f161e8700 1 -- 172.21.15.89:0/2742942710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0035b0 con 0x7f5efc003a30 2023-07-06T02:23:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f169e9700 1 -- 172.21.15.89:0/2742942710 reap_dead start 2023-07-06T02:23:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f161e8700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc003a30 0x7f5efc003e40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5f00007fd0 tx=0x7f5efc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7e6758376f47a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000052d0 con 0x7f5efc003a30 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.642+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f00005450 con 0x7f5efc003a30 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f151e6700 1 -- 172.21.15.89:0/2742942710 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000038c0 con 0x7f5efc003a30 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f1844c700 1 -- 172.21.15.89:0/2742942710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc003a30 msgr2=0x7f5efc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f1844c700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc003a30 0x7f5efc003e40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5f00007fd0 tx=0x7f5efc005880 comp rx=0 tx=0).stop 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f1844c700 1 -- 172.21.15.89:0/2742942710 shutdown_connections 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f1844c700 1 --2- 172.21.15.89:0/2742942710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc003a30 0x7f5efc003e40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f1844c700 1 -- 172.21.15.89:0/2742942710 >> 172.21.15.89:0/2742942710 conn(0x7f5f100d0b00 msgr2=0x7f5f100d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:22.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f1844c700 1 -- 172.21.15.89:0/2742942710 shutdown_connections 2023-07-06T02:23:22.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.643+0000 7f5f1844c700 1 -- 172.21.15.89:0/2742942710 wait complete. 2023-07-06T02:23:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.644+0000 7f5f1844c700 1 Processor -- start 2023-07-06T02:23:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.644+0000 7f5f1844c700 1 -- start start 2023-07-06T02:23:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f1844c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a30 0x7f5f100c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f1844c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c4b00 0x7f5f101574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f1844c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f100c9e60 0x7f5f100c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f1844c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1015a320 con 0x7f5f100c4b00 2023-07-06T02:23:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f1844c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100c8dc0 con 0x7f5efc003a30 2023-07-06T02:23:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f1844c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100c90a0 con 0x7f5f100c9e60 2023-07-06T02:23:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f159e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c4b00 0x7f5f101574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f159e7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c4b00 0x7f5f101574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33140/0 (socket says 172.21.15.89:33140) 2023-07-06T02:23:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f159e7700 1 -- 172.21.15.89:0/4048547041 learned_addr learned my addr 172.21.15.89:0/4048547041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f161e8700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a30 0x7f5f100c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.645+0000 7f5f169e9700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f100c9e60 0x7f5f100c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f159e7700 1 -- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f100c9e60 msgr2=0x7f5f100c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f159e7700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f100c9e60 0x7f5f100c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f159e7700 1 -- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a30 msgr2=0x7f5f100c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f159e7700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a30 0x7f5f100c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f159e7700 1 -- 172.21.15.89:0/4048547041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100c9380 con 0x7f5f100c4b00 2023-07-06T02:23:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f161e8700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a30 0x7f5f100c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f159e7700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c4b00 0x7f5f101574e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c011aa0 tx=0x7f5f0c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f06ffd700 1 -- 172.21.15.89:0/4048547041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00f980 con 0x7f5f100c4b00 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f06ffd700 1 -- 172.21.15.89:0/4048547041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c00fb00 con 0x7f5f100c4b00 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f100c9660 con 0x7f5f100c4b00 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f06ffd700 1 -- 172.21.15.89:0/4048547041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00feb0 con 0x7f5f100c4b00 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.646+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f100ca2d0 con 0x7f5f100c4b00 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.647+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f100c9b80 con 0x7f5f100c4b00 2023-07-06T02:23:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.647+0000 7f5f06ffd700 1 -- 172.21.15.89:0/4048547041 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f0c014440 con 0x7f5f100c4b00 2023-07-06T02:23:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.648+0000 7f5f06ffd700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ad10 0x7f5ef406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.648+0000 7f5f06ffd700 1 -- 172.21.15.89:0/4048547041 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5f0c089e50 con 0x7f5f100c4b00 2023-07-06T02:23:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.648+0000 7f5f161e8700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ad10 0x7f5ef406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.648+0000 7f5f161e8700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ad10 0x7f5ef406d1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f00005040 tx=0x7f5f00009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.652+0000 7f5f06ffd700 1 -- 172.21.15.89:0/4048547041 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f100c9b80 con 0x7f5f100c4b00 2023-07-06T02:23:22.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.840+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f100c9b80 con 0x7f5f100c4b00 2023-07-06T02:23:22.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.840+0000 7f5f06ffd700 1 -- 172.21.15.89:0/4048547041 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f100c9b80 con 0x7f5f100c4b00 2023-07-06T02:23:22.843 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:22.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ad10 msgr2=0x7f5ef406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ad10 0x7f5ef406d1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f00005040 tx=0x7f5f00009030 comp rx=0 tx=0).stop 2023-07-06T02:23:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c4b00 msgr2=0x7f5f101574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c4b00 0x7f5f101574e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c011aa0 tx=0x7f5f0c011e60 comp rx=0 tx=0).stop 2023-07-06T02:23:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 shutdown_connections 2023-07-06T02:23:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f100c9e60 0x7f5f100c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a30 0x7f5f100c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ad10 0x7f5ef406d1c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 --2- 172.21.15.89:0/4048547041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c4b00 0x7f5f101574e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 >> 172.21.15.89:0/4048547041 conn(0x7f5f100d0b00 msgr2=0x7f5f1015d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:22.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 shutdown_connections 2023-07-06T02:23:22.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.842+0000 7f5f1844c700 1 -- 172.21.15.89:0/4048547041 wait complete. 2023-07-06T02:23:22.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.978+0000 7f0f2e45a700 1 Processor -- start 2023-07-06T02:23:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.978+0000 7f0f2e45a700 1 -- start start 2023-07-06T02:23:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.979+0000 7f0f2e45a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 0x7f0f280b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.979+0000 7f0f2e45a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f280b5960 con 0x7f0f28070920 2023-07-06T02:23:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.979+0000 7f0f2e45a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f280cf4a0 con 0x7f0f280ccfe0 2023-07-06T02:23:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.979+0000 7f0f2e45a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280b38d0 con 0x7f0f2815c4f0 2023-07-06T02:23:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.979+0000 7f0f2c9f7700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0f28070920 0x7f0f280cb5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58052/0 (socket says 172.21.15.89:58052) 2023-07-06T02:23:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.979+0000 7f0f2c9f7700 1 -- 172.21.15.89:0/875965514 learned_addr learned my addr 172.21.15.89:0/875965514 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f27fff700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 0x7f0f280b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695697513 0 0) 0x7f0f280b5960 con 0x7f0f28070920 2023-07-06T02:23:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f100035b0 con 0x7f0f28070920 2023-07-06T02:23:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 751405274 0 0) 0x7f0f100035b0 con 0x7f0f28070920 2023-07-06T02:23:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 msgr2=0x7f0f280b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 0x7f0f280b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 >> v1:172.21.15.172:6789/0 conn(0x7f0f280ccfe0 legacy=0x7f0f28070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280b3bb0 con 0x7f0f28070920 2023-07-06T02:23:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f0f200019f0 con 0x7f0f28070920 2023-07-06T02:23:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 >> v1:172.21.15.89:6789/0 conn(0x7f0f28070920 legacy=0x7f0f280cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f10003a30 0x7f0f10003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 0x7f0f10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280b38d0 con 0x7f0f10003a30 2023-07-06T02:23:22.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1000bbd0 con 0x7f0f10004d90 2023-07-06T02:23:22.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1000bd90 con 0x7f0f2815c4f0 2023-07-06T02:23:22.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f27fff700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f10003a30 0x7f0f10003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f277fe700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 0x7f0f10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f2c9f7700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f27fff700 1 -- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 msgr2=0x7f0f10004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f27fff700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 0x7f0f10004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f27fff700 1 -- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f10004d90 msgr2=0x7f0f10005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f27fff700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f27fff700 1 -- 172.21.15.89:0/875965514 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100035b0 con 0x7f0f10003a30 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f277fe700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2815c4f0 0x7f0f10004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.980+0000 7f0f2c9f7700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:22.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f27fff700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f10003a30 0x7f0f10003e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009890 tx=0x7f0f10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6dbccec957f7ae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00c070 con 0x7f0f10003a30 2023-07-06T02:23:22.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c013610 con 0x7f0f10003a30 2023-07-06T02:23:22.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f26ffd700 1 -- 172.21.15.89:0/875965514 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c0139a0 con 0x7f0f10003a30 2023-07-06T02:23:22.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f2e45a700 1 -- 172.21.15.89:0/875965514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f10003a30 msgr2=0x7f0f10003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f2e45a700 1 --2- 172.21.15.89:0/875965514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f10003a30 0x7f0f10003e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009890 tx=0x7f0f10005880 comp rx=0 tx=0).stop 2023-07-06T02:23:22.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f2c9f7700 1 -- 172.21.15.89:0/875965514 reap_dead start 2023-07-06T02:23:22.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f2e45a700 1 -- 172.21.15.89:0/875965514 shutdown_connections 2023-07-06T02:23:22.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f2e45a700 1 -- 172.21.15.89:0/875965514 >> 172.21.15.89:0/875965514 conn(0x7f0f280d09e0 msgr2=0x7f0f280d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:22.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f2e45a700 1 -- 172.21.15.89:0/875965514 shutdown_connections 2023-07-06T02:23:22.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.981+0000 7f0f2e45a700 1 -- 172.21.15.89:0/875965514 wait complete. 2023-07-06T02:23:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.982+0000 7f0f2e45a700 1 Processor -- start 2023-07-06T02:23:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.983+0000 7f0f2e45a700 1 -- start start 2023-07-06T02:23:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.983+0000 7f0f2e45a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f10003a30 0x7f0f280af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.983+0000 7f0f2e45a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f281563a0 0x7f0f28155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.983+0000 7f0f2e45a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f280ab8f0 0x7f0f280abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.983+0000 7f0f2e45a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280ac6e0 con 0x7f0f280ab8f0 2023-07-06T02:23:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.983+0000 7f0f2e45a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280ac2a0 con 0x7f0f281563a0 2023-07-06T02:23:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.983+0000 7f0f2e45a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280c6a40 con 0x7f0f10003a30 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f280ab8f0 0x7f0f280abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f280ab8f0 0x7f0f280abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33162/0 (socket says 172.21.15.89:33162) 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 -- 172.21.15.89:0/2928757812 learned_addr learned my addr 172.21.15.89:0/2928757812 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f27fff700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f10003a30 0x7f0f280af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f277fe700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f281563a0 0x7f0f28155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 -- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f10003a30 msgr2=0x7f0f280af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f10003a30 0x7f0f280af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 -- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f281563a0 msgr2=0x7f0f28155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f281563a0 0x7f0f28155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 -- 172.21.15.89:0/2928757812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280afae0 con 0x7f0f280ab8f0 2023-07-06T02:23:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f277fe700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f281563a0 0x7f0f28155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2c9f7700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f280ab8f0 0x7f0f280abd60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0f28070d10 tx=0x7f0f20007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f24ff9700 1 -- 172.21.15.89:0/2928757812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20017070 con 0x7f0f280ab8f0 2023-07-06T02:23:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f24ff9700 1 -- 172.21.15.89:0/2928757812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2000ebe0 con 0x7f0f280ab8f0 2023-07-06T02:23:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f28156170 con 0x7f0f280ab8f0 2023-07-06T02:23:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f24ff9700 1 -- 172.21.15.89:0/2928757812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000edb0 con 0x7f0f280ab8f0 2023-07-06T02:23:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.984+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f280c63f0 con 0x7f0f280ab8f0 2023-07-06T02:23:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.985+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28074270 con 0x7f0f280ab8f0 2023-07-06T02:23:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.985+0000 7f0f24ff9700 1 -- 172.21.15.89:0/2928757812 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f20013420 con 0x7f0f280ab8f0 2023-07-06T02:23:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.986+0000 7f0f24ff9700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0f1406ace0 0x7f0f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.986+0000 7f0f24ff9700 1 -- 172.21.15.89:0/2928757812 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0f200894b0 con 0x7f0f280ab8f0 2023-07-06T02:23:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.986+0000 7f0f27fff700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0f1406ace0 0x7f0f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.986+0000 7f0f27fff700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0f1406ace0 0x7f0f1406d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c016040 tx=0x7f0f1c01f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:22.990+0000 7f0f24ff9700 1 -- 172.21.15.89:0/2928757812 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0f20054330 con 0x7f0f280ab8f0 2023-07-06T02:23:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.174+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0f280d0080 con 0x7f0f280ab8f0 2023-07-06T02:23:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.175+0000 7f0f24ff9700 1 -- 172.21.15.89:0/2928757812 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0f20057950 con 0x7f0f280ab8f0 2023-07-06T02:23:23.178 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0f1406ace0 msgr2=0x7f0f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0f1406ace0 0x7f0f1406d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c016040 tx=0x7f0f1c01f040 comp rx=0 tx=0).stop 2023-07-06T02:23:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f280ab8f0 msgr2=0x7f0f280abd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f280ab8f0 0x7f0f280abd60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0f28070d10 tx=0x7f0f20007c90 comp rx=0 tx=0).stop 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 shutdown_connections 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0f1406ace0 0x7f0f1406d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f280ab8f0 0x7f0f280abd60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f10003a30 0x7f0f280af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 --2- 172.21.15.89:0/2928757812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f281563a0 0x7f0f28155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 >> 172.21.15.89:0/2928757812 conn(0x7f0f280d09e0 msgr2=0x7f0f280b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 shutdown_connections 2023-07-06T02:23:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.177+0000 7f0f2e45a700 1 -- 172.21.15.89:0/2928757812 wait complete. 2023-07-06T02:23:23.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:23.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.308+0000 7fb7c8d32700 1 Processor -- start 2023-07-06T02:23:23.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.308+0000 7fb7c8d32700 1 -- start start 2023-07-06T02:23:23.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c8d32700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7c415c610 0x7fb7c40cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c8d32700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c8d32700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c40cb150 con 0x7fb7c40b5e50 2023-07-06T02:23:23.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c8d32700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c40cb430 con 0x7fb7c415c610 2023-07-06T02:23:23.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c8d32700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40b6310 con 0x7fb7c4070a40 2023-07-06T02:23:23.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7c415c610 0x7fb7c40cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c259c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7c415c610 0x7fb7c40cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33166/0 (socket says 172.21.15.89:33166) 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c259c700 1 -- 172.21.15.89:0/503609916 learned_addr learned my addr 172.21.15.89:0/503609916 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c2d9d700 1 --2- 172.21.15.89:0/503609916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.309+0000 7fb7c259c700 1 -- 172.21.15.89:0/503609916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 msgr2=0x7fb7c40cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c259c700 1 --2- 172.21.15.89:0/503609916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c259c700 1 -- 172.21.15.89:0/503609916 >> v1:172.21.15.172:6789/0 conn(0x7fb7c40b5e50 legacy=0x7fb7c4070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c259c700 1 -- 172.21.15.89:0/503609916 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c40b65f0 con 0x7fb7c415c610 2023-07-06T02:23:23.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c2d9d700 1 --2- 172.21.15.89:0/503609916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:23.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c259c700 1 --2- 172.21.15.89:0/503609916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c415c610 0x7fb7c40cfed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009e50 tx=0x7fb7bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb473f68b80e2eb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:23.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c159a700 1 -- 172.21.15.89:0/503609916 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00e070 con 0x7fb7c415c610 2023-07-06T02:23:23.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c159a700 1 -- 172.21.15.89:0/503609916 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc003a20 con 0x7fb7c415c610 2023-07-06T02:23:23.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c159a700 1 -- 172.21.15.89:0/503609916 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc003dd0 con 0x7fb7c415c610 2023-07-06T02:23:23.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c8d32700 1 -- 172.21.15.89:0/503609916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c415c610 msgr2=0x7fb7c40cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/503609916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c415c610 0x7fb7c40cfed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009e50 tx=0x7fb7bc00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c8d32700 1 -- 172.21.15.89:0/503609916 shutdown_connections 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/503609916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/503609916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c415c610 0x7fb7c40cfed0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c8d32700 1 -- 172.21.15.89:0/503609916 >> 172.21.15.89:0/503609916 conn(0x7fb7c40d0b00 msgr2=0x7fb7c40d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.310+0000 7fb7c8d32700 1 -- 172.21.15.89:0/503609916 shutdown_connections 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.311+0000 7fb7c8d32700 1 -- 172.21.15.89:0/503609916 wait complete. 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.312+0000 7fb7c8d32700 1 Processor -- start 2023-07-06T02:23:23.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.312+0000 7fb7c8d32700 1 -- start start 2023-07-06T02:23:23.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c8d32700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c8d32700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40b5e50 0x7fb7c415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c8d32700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c415c610 0x7fb7c40c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c8d32700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415e450 con 0x7fb7c40b5e50 2023-07-06T02:23:23.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c8d32700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415d930 con 0x7fb7c415c610 2023-07-06T02:23:23.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c8d32700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415dbb0 con 0x7fb7c4070a40 2023-07-06T02:23:23.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40b5e50 0x7fb7c415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40b5e50 0x7fb7c415d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33174/0 (socket says 172.21.15.89:33174) 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c1d9b700 1 -- 172.21.15.89:0/1086012096 learned_addr learned my addr 172.21.15.89:0/1086012096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c259c700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.313+0000 7fb7c2d9d700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c415c610 0x7fb7c40c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c1d9b700 1 -- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 msgr2=0x7fb7c40c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c1d9b700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c1d9b700 1 -- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c415c610 msgr2=0x7fb7c40c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c1d9b700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c415c610 0x7fb7c40c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c1d9b700 1 -- 172.21.15.89:0/1086012096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc009b30 con 0x7fb7c40b5e50 2023-07-06T02:23:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c1d9b700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40b5e50 0x7fb7c415d3f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00c500 tx=0x7fb7ac00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7baffd700 1 -- 172.21.15.89:0/1086012096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac011790 con 0x7fb7c40b5e50 2023-07-06T02:23:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7baffd700 1 -- 172.21.15.89:0/1086012096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0118f0 con 0x7fb7c40b5e50 2023-07-06T02:23:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c415de90 con 0x7fb7c40b5e50 2023-07-06T02:23:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7baffd700 1 -- 172.21.15.89:0/1086012096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac011ca0 con 0x7fb7c40b5e50 2023-07-06T02:23:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.314+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c41e5f10 con 0x7fb7c40b5e50 2023-07-06T02:23:23.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.315+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c4074270 con 0x7fb7c40b5e50 2023-07-06T02:23:23.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.315+0000 7fb7baffd700 1 -- 172.21.15.89:0/1086012096 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7ac01c460 con 0x7fb7c40b5e50 2023-07-06T02:23:23.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.316+0000 7fb7baffd700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7a406ace0 0x7fb7a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.316+0000 7fb7baffd700 1 -- 172.21.15.89:0/1086012096 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb7ac089410 con 0x7fb7c40b5e50 2023-07-06T02:23:23.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.316+0000 7fb7c259c700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7a406ace0 0x7fb7a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.316+0000 7fb7c259c700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7a406ace0 0x7fb7a406d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc015040 tx=0x7fb7bc009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:23.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.320+0000 7fb7baffd700 1 -- 172.21.15.89:0/1086012096 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb7ac054260 con 0x7fb7c40b5e50 2023-07-06T02:23:23.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.549+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7c41e60d0 con 0x7fb7c40b5e50 2023-07-06T02:23:23.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.550+0000 7fb7baffd700 1 -- 172.21.15.89:0/1086012096 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fb7ac057880 con 0x7fb7c40b5e50 2023-07-06T02:23:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:23.553 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:23.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7a406ace0 msgr2=0x7fb7a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7a406ace0 0x7fb7a406d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc015040 tx=0x7fb7bc009000 comp rx=0 tx=0).stop 2023-07-06T02:23:23.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40b5e50 msgr2=0x7fb7c415d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40b5e50 0x7fb7c415d3f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00c500 tx=0x7fb7ac00c810 comp rx=0 tx=0).stop 2023-07-06T02:23:23.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 shutdown_connections 2023-07-06T02:23:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7c415c610 0x7fb7c40c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7c4070a40 0x7fb7c40c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb7a406ace0 0x7fb7a406d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 --2- 172.21.15.89:0/1086012096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7c40b5e50 0x7fb7c415d3f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 >> 172.21.15.89:0/1086012096 conn(0x7fb7c40d0b00 msgr2=0x7fb7c40d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:23.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 shutdown_connections 2023-07-06T02:23:23.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.552+0000 7fb7c8d32700 1 -- 172.21.15.89:0/1086012096 wait complete. 2023-07-06T02:23:23.557 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:23.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.684+0000 7f34797f5700 1 Processor -- start 2023-07-06T02:23:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.684+0000 7f34797f5700 1 -- start start 2023-07-06T02:23:23.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34797f5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3474070f20 0x7f34740713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34797f5700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3474160000 0x7f3474160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34797f5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34740cb9f0 con 0x7f347415c500 2023-07-06T02:23:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34797f5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3474071900 con 0x7f3474070f20 2023-07-06T02:23:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34797f5700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347415d8e0 con 0x7f3474160000 2023-07-06T02:23:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3474070f20 0x7f34740713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3474070f20 0x7f34740713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33182/0 (socket says 172.21.15.89:33182) 2023-07-06T02:23:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 -- 172.21.15.89:0/3566588024 learned_addr learned my addr 172.21.15.89:0/3566588024 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f3472ffd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f347415c500 0x7f347406edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58082/0 (socket says 172.21.15.89:58082) 2023-07-06T02:23:23.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34737fe700 1 --2- 172.21.15.89:0/3566588024 >> v2:172.21.15.172:3300/0 conn(0x7f3474160000 0x7f3474160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 -- 172.21.15.89:0/3566588024 >> v2:172.21.15.172:3300/0 conn(0x7f3474160000 msgr2=0x7f3474160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:23.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 --2- 172.21.15.89:0/3566588024 >> v2:172.21.15.172:3300/0 conn(0x7f3474160000 0x7f3474160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 -- 172.21.15.89:0/3566588024 >> v1:172.21.15.89:6789/0 conn(0x7f347415c500 legacy=0x7f347406edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f3471ffb700 1 -- 172.21.15.89:0/3566588024 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990756566 0 0) 0x7f34740cb9f0 con 0x7f347415c500 2023-07-06T02:23:23.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 -- 172.21.15.89:0/3566588024 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474160960 con 0x7f3474070f20 2023-07-06T02:23:23.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34737fe700 1 --2- 172.21.15.89:0/3566588024 >> v2:172.21.15.172:3300/0 conn(0x7f3474160000 0x7f3474160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:23.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.685+0000 7f34727fc700 1 --2- 172.21.15.89:0/3566588024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474070f20 0x7f34740713c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f347415db40 tx=0x7f3464009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f5933b31546529 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:23.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f3471ffb700 1 -- 172.21.15.89:0/3566588024 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400b870 con 0x7f3474070f20 2023-07-06T02:23:23.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f3471ffb700 1 -- 172.21.15.89:0/3566588024 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346400b9d0 con 0x7f3474070f20 2023-07-06T02:23:23.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f3471ffb700 1 -- 172.21.15.89:0/3566588024 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400bd60 con 0x7f3474070f20 2023-07-06T02:23:23.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 -- 172.21.15.89:0/3566588024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474070f20 msgr2=0x7f34740713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 --2- 172.21.15.89:0/3566588024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474070f20 0x7f34740713c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f347415db40 tx=0x7f3464009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 -- 172.21.15.89:0/3566588024 shutdown_connections 2023-07-06T02:23:23.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 --2- 172.21.15.89:0/3566588024 >> v2:172.21.15.172:3300/0 conn(0x7f3474160000 0x7f3474160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 --2- 172.21.15.89:0/3566588024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474070f20 0x7f34740713c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 -- 172.21.15.89:0/3566588024 >> 172.21.15.89:0/3566588024 conn(0x7f34740cf1c0 msgr2=0x7f34740cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:23.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 -- 172.21.15.89:0/3566588024 shutdown_connections 2023-07-06T02:23:23.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.686+0000 7f34797f5700 1 -- 172.21.15.89:0/3566588024 wait complete. 2023-07-06T02:23:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.687+0000 7f34797f5700 1 Processor -- start 2023-07-06T02:23:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34797f5700 1 -- start start 2023-07-06T02:23:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34797f5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3474070f20 0x7f347415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34797f5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415c500 0x7f34740c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34797f5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474160000 0x7f34740c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34797f5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34740c4b30 con 0x7f3474160000 2023-07-06T02:23:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34797f5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347415ad40 con 0x7f3474070f20 2023-07-06T02:23:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34797f5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34740aec60 con 0x7f347415c500 2023-07-06T02:23:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474160000 0x7f34740c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474160000 0x7f34740c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33190/0 (socket says 172.21.15.89:33190) 2023-07-06T02:23:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.688+0000 7f34737fe700 1 -- 172.21.15.89:0/3602161304 learned_addr learned my addr 172.21.15.89:0/3602161304 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34727fc700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415c500 0x7f34740c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f3472ffd700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3474070f20 0x7f347415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34737fe700 1 -- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415c500 msgr2=0x7f34740c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34737fe700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415c500 0x7f34740c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34737fe700 1 -- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3474070f20 msgr2=0x7f347415a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34737fe700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3474070f20 0x7f347415a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34737fe700 1 -- 172.21.15.89:0/3602161304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464009550 con 0x7f3474160000 2023-07-06T02:23:23.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34737fe700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474160000 0x7f34740c4fc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f346c00cd50 tx=0x7f346c00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:23.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f345b7fe700 1 -- 172.21.15.89:0/3602161304 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00f480 con 0x7f3474160000 2023-07-06T02:23:23.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f345b7fe700 1 -- 172.21.15.89:0/3602161304 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c00f600 con 0x7f3474160000 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34740aef40 con 0x7f3474160000 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f345b7fe700 1 -- 172.21.15.89:0/3602161304 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00f9b0 con 0x7f3474160000 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.689+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34740c9b30 con 0x7f3474160000 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.690+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34740c4ca0 con 0x7f3474160000 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.690+0000 7f345b7fe700 1 -- 172.21.15.89:0/3602161304 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f346c00b040 con 0x7f3474160000 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.691+0000 7f345b7fe700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.691+0000 7f3472ffd700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:23.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.691+0000 7f345b7fe700 1 -- 172.21.15.89:0/3602161304 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f346c089f70 con 0x7f3474160000 2023-07-06T02:23:23.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.691+0000 7f3472ffd700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f346006ace0 0x7f346006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3468000f60 tx=0x7f3468007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:23.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.695+0000 7f345b7fe700 1 -- 172.21.15.89:0/3602161304 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f346c0bc310 con 0x7f3474160000 2023-07-06T02:23:23.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.885+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3474071dd0 con 0x7f3474160000 2023-07-06T02:23:23.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.886+0000 7f345b7fe700 1 -- 172.21.15.89:0/3602161304 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f346c054dc0 con 0x7f3474160000 2023-07-06T02:23:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:23.890 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f346006ace0 msgr2=0x7f346006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f346006ace0 0x7f346006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3468000f60 tx=0x7f3468007be0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474160000 msgr2=0x7f34740c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474160000 0x7f34740c4fc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f346c00cd50 tx=0x7f346c00a560 comp rx=0 tx=0).stop 2023-07-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 shutdown_connections 2023-07-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3474160000 0x7f34740c4fc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3474070f20 0x7f347415a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 --2- 172.21.15.89:0/3602161304 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415c500 0x7f34740c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 >> 172.21.15.89:0/3602161304 conn(0x7f34740cf1c0 msgr2=0x7f347415b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 shutdown_connections 2023-07-06T02:23:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:23.888+0000 7f34797f5700 1 -- 172.21.15.89:0/3602161304 wait complete. 2023-07-06T02:23:23.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:24.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 Processor -- start 2023-07-06T02:23:24.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 -- start start 2023-07-06T02:23:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f21d4159fd0 0x7f21d40cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d4160b80 0x7f21d406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f21d406c410 0x7f21d406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d406c890 con 0x7f21d406c410 2023-07-06T02:23:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d4161020 con 0x7f21d4159fd0 2023-07-06T02:23:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d9d21700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40cdb60 con 0x7f21d4160b80 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f21d406c410 0x7f21d406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.021+0000 7f21d3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f21d406c410 0x7f21d406a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33198/0 (socket says 172.21.15.89:33198) 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d3fff700 1 -- 172.21.15.89:0/2264120083 learned_addr learned my addr 172.21.15.89:0/2264120083 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d37fe700 1 --2- 172.21.15.89:0/2264120083 >> v2:172.21.15.172:3300/0 conn(0x7f21d4159fd0 0x7f21d40cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d2ffd700 1 --2- 172.21.15.89:0/2264120083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d4160b80 0x7f21d406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d3fff700 1 -- 172.21.15.89:0/2264120083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d4160b80 msgr2=0x7f21d406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d3fff700 1 --2- 172.21.15.89:0/2264120083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d4160b80 0x7f21d406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d3fff700 1 -- 172.21.15.89:0/2264120083 >> v2:172.21.15.172:3300/0 conn(0x7f21d4159fd0 msgr2=0x7f21d40cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d3fff700 1 --2- 172.21.15.89:0/2264120083 >> v2:172.21.15.172:3300/0 conn(0x7f21d4159fd0 0x7f21d40cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d3fff700 1 -- 172.21.15.89:0/2264120083 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d415ce60 con 0x7f21d406c410 2023-07-06T02:23:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d2ffd700 1 --2- 172.21.15.89:0/2264120083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d4160b80 0x7f21d406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d3fff700 1 --2- 172.21.15.89:0/2264120083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d406c410 0x7f21d406a060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009e50 tx=0x7f21cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efd503bb54a647 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d27fc700 1 -- 172.21.15.89:0/2264120083 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00e070 con 0x7f21d406c410 2023-07-06T02:23:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d27fc700 1 -- 172.21.15.89:0/2264120083 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc00b8d0 con 0x7f21d406c410 2023-07-06T02:23:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.022+0000 7f21d27fc700 1 -- 172.21.15.89:0/2264120083 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00bc80 con 0x7f21d406c410 2023-07-06T02:23:24.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 -- 172.21.15.89:0/2264120083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d406c410 msgr2=0x7f21d406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 --2- 172.21.15.89:0/2264120083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d406c410 0x7f21d406a060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009e50 tx=0x7f21cc00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:24.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 -- 172.21.15.89:0/2264120083 shutdown_connections 2023-07-06T02:23:24.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 --2- 172.21.15.89:0/2264120083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d406c410 0x7f21d406a060 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 --2- 172.21.15.89:0/2264120083 >> v2:172.21.15.172:3300/0 conn(0x7f21d4159fd0 0x7f21d40cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 --2- 172.21.15.89:0/2264120083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d4160b80 0x7f21d406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 -- 172.21.15.89:0/2264120083 >> 172.21.15.89:0/2264120083 conn(0x7f21d40d0590 msgr2=0x7f21d40d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 -- 172.21.15.89:0/2264120083 shutdown_connections 2023-07-06T02:23:24.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.023+0000 7f21d9d21700 1 -- 172.21.15.89:0/2264120083 wait complete. 2023-07-06T02:23:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.024+0000 7f21d9d21700 1 Processor -- start 2023-07-06T02:23:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.024+0000 7f21d9d21700 1 -- start start 2023-07-06T02:23:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d9d21700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406c410 0x7f21d40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d9d21700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d4159fd0 0x7f21d40b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d9d21700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d4160b80 0x7f21d40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d9d21700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40ca080 con 0x7f21d4159fd0 2023-07-06T02:23:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d9d21700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40b7ce0 con 0x7f21d4160b80 2023-07-06T02:23:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d9d21700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40b7fc0 con 0x7f21d406c410 2023-07-06T02:23:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d4159fd0 0x7f21d40b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d4159fd0 0x7f21d40b7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33202/0 (socket says 172.21.15.89:33202) 2023-07-06T02:23:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 -- 172.21.15.89:0/579452174 learned_addr learned my addr 172.21.15.89:0/579452174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d37fe700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406c410 0x7f21d40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d3fff700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d4160b80 0x7f21d40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 -- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406c410 msgr2=0x7f21d40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406c410 0x7f21d40cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 -- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d4160b80 msgr2=0x7f21d40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d4160b80 0x7f21d40c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 -- 172.21.15.89:0/579452174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009b30 con 0x7f21d4159fd0 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d3fff700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d4160b80 0x7f21d40c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d2ffd700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d4159fd0 0x7f21d40b7290 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00e810 tx=0x7f21bc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21cbfff700 1 -- 172.21.15.89:0/579452174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc00c020 con 0x7f21d4159fd0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21cbfff700 1 -- 172.21.15.89:0/579452174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc004250 con 0x7f21d4159fd0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.025+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d40b82a0 con 0x7f21d4159fd0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.026+0000 7f21cbfff700 1 -- 172.21.15.89:0/579452174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0036a0 con 0x7f21d4159fd0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.026+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d414e4b0 con 0x7f21d4159fd0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.027+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d40ca1f0 con 0x7f21d4159fd0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.027+0000 7f21cbfff700 1 -- 172.21.15.89:0/579452174 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21bc003800 con 0x7f21d4159fd0 2023-07-06T02:23:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.028+0000 7f21cbfff700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f21b406ace0 0x7f21b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.028+0000 7f21d37fe700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f21b406ace0 0x7f21b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.028+0000 7f21cbfff700 1 -- 172.21.15.89:0/579452174 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f21bc0891e0 con 0x7f21d4159fd0 2023-07-06T02:23:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.028+0000 7f21d37fe700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f21b406ace0 0x7f21b406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009b60 tx=0x7f21c4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.032+0000 7f21cbfff700 1 -- 172.21.15.89:0/579452174 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21bc053fa0 con 0x7f21d4159fd0 2023-07-06T02:23:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.263+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21d4074270 con 0x7f21d4159fd0 2023-07-06T02:23:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.263+0000 7f21cbfff700 1 -- 172.21.15.89:0/579452174 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f21bc0575c0 con 0x7f21d4159fd0 2023-07-06T02:23:24.267 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:24.267 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f21b406ace0 msgr2=0x7f21b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f21b406ace0 0x7f21b406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009b60 tx=0x7f21c4009210 comp rx=0 tx=0).stop 2023-07-06T02:23:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d4159fd0 msgr2=0x7f21d40b7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d4159fd0 0x7f21d40b7290 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00e810 tx=0x7f21bc00eb20 comp rx=0 tx=0).stop 2023-07-06T02:23:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 shutdown_connections 2023-07-06T02:23:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d4160b80 0x7f21d40c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406c410 0x7f21d40cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f21b406ace0 0x7f21b406d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 --2- 172.21.15.89:0/579452174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21d4159fd0 0x7f21d40b7290 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 >> 172.21.15.89:0/579452174 conn(0x7f21d40d0590 msgr2=0x7f21d40d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 shutdown_connections 2023-07-06T02:23:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.266+0000 7f21d9d21700 1 -- 172.21.15.89:0/579452174 wait complete. 2023-07-06T02:23:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:24.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T02:22:52.797894+0000', 'modified': '2023-07-06T02:23:11.177784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/1524050092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 1524050092}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 1524050092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T02:23:24.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T02:23:24.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T02:23:24.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.804+0000 7f553191d700 1 Processor -- start 2023-07-06T02:23:24.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.804+0000 7f553191d700 1 -- start start 2023-07-06T02:23:24.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f553191d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 0x7f552c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f553191d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f552c06b220 0x7f552c15de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f553191d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552c06a990 con 0x7f552c0cc000 2023-07-06T02:23:24.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f553191d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552c06ac70 con 0x7f552c06b220 2023-07-06T02:23:24.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f553191d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c06d170 con 0x7f552c0b1490 2023-07-06T02:23:24.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f552a7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f552c0cc000 0x7f552c0b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58110/0 (socket says 172.21.15.89:58110) 2023-07-06T02:23:24.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f552a7fc700 1 -- 172.21.15.89:0/3101788356 learned_addr learned my addr 172.21.15.89:0/3101788356 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:24.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806114602 0 0) 0x7f552c06a990 con 0x7f552c0cc000 2023-07-06T02:23:24.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f552affd700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 0x7f552c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.805+0000 7f552b7fe700 1 --2- 172.21.15.89:0/3101788356 >> v2:172.21.15.172:3300/0 conn(0x7f552c06b220 0x7f552c15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55140035b0 con 0x7f552c0cc000 2023-07-06T02:23:24.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2524195412 0 0) 0x7f55140035b0 con 0x7f552c0cc000 2023-07-06T02:23:24.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 msgr2=0x7f552c0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 0x7f552c0caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 >> v2:172.21.15.172:3300/0 conn(0x7f552c06b220 msgr2=0x7f552c15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 --2- 172.21.15.89:0/3101788356 >> v2:172.21.15.172:3300/0 conn(0x7f552c06b220 0x7f552c15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c06ce60 con 0x7f552c0cc000 2023-07-06T02:23:24.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f551c001670 con 0x7f552c0cc000 2023-07-06T02:23:24.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 >> v1:172.21.15.89:6789/0 conn(0x7f552c0cc000 legacy=0x7f552c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5514003a00 0x7f5514003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5514004430 0x7f55140048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 0x7f5514004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c06ac70 con 0x7f5514004430 2023-07-06T02:23:24.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c06d170 con 0x7f5514003a00 2023-07-06T02:23:24.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551400ba10 con 0x7f552c0b1490 2023-07-06T02:23:24.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552a7fc700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5514004430 0x7f55140048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552b7fe700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 0x7f5514004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552affd700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5514003a00 0x7f5514003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552a7fc700 1 -- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 msgr2=0x7f5514004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552a7fc700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f552c0b1490 0x7f5514004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552a7fc700 1 -- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5514003a00 msgr2=0x7f5514003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552a7fc700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5514003a00 0x7f5514003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552a7fc700 1 -- 172.21.15.89:0/3101788356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140035b0 con 0x7f5514004430 2023-07-06T02:23:24.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.806+0000 7f552a7fc700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5514004430 0x7f55140048f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f551c00c370 tx=0x7f5514005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18979130ab1c660b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c007c60 con 0x7f5514004430 2023-07-06T02:23:24.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c005960 con 0x7f5514004430 2023-07-06T02:23:24.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f5529ffb700 1 -- 172.21.15.89:0/3101788356 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c005d10 con 0x7f5514004430 2023-07-06T02:23:24.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f553191d700 1 -- 172.21.15.89:0/3101788356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5514004430 msgr2=0x7f55140048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f553191d700 1 --2- 172.21.15.89:0/3101788356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5514004430 0x7f55140048f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f551c00c370 tx=0x7f5514005880 comp rx=0 tx=0).stop 2023-07-06T02:23:24.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f552b7fe700 1 -- 172.21.15.89:0/3101788356 reap_dead start 2023-07-06T02:23:24.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f553191d700 1 -- 172.21.15.89:0/3101788356 shutdown_connections 2023-07-06T02:23:24.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f553191d700 1 -- 172.21.15.89:0/3101788356 >> 172.21.15.89:0/3101788356 conn(0x7f552c0cc490 msgr2=0x7f552c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f553191d700 1 -- 172.21.15.89:0/3101788356 shutdown_connections 2023-07-06T02:23:24.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.807+0000 7f553191d700 1 -- 172.21.15.89:0/3101788356 wait complete. 2023-07-06T02:23:24.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.808+0000 7f553191d700 1 Processor -- start 2023-07-06T02:23:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f553191d700 1 -- start start 2023-07-06T02:23:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f553191d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5514004430 0x7f552c0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f553191d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552c0ca4a0 0x7f552c0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f553191d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c1e50e0 0x7f552c1e5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f553191d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c1e6700 con 0x7f552c0ca4a0 2023-07-06T02:23:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f553191d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0cae50 con 0x7f552c1e50e0 2023-07-06T02:23:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f553191d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c1e5e60 con 0x7f5514004430 2023-07-06T02:23:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552c0ca4a0 0x7f552c0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552c0ca4a0 0x7f552c0ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33222/0 (socket says 172.21.15.89:33222) 2023-07-06T02:23:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f552a7fc700 1 -- 172.21.15.89:0/105051811 learned_addr learned my addr 172.21.15.89:0/105051811 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.809+0000 7f552affd700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5514004430 0x7f552c0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f552b7fe700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c1e50e0 0x7f552c1e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f552a7fc700 1 -- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5514004430 msgr2=0x7f552c0c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f552a7fc700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5514004430 0x7f552c0c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f552a7fc700 1 -- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c1e50e0 msgr2=0x7f552c1e5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f552a7fc700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c1e50e0 0x7f552c1e5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f552a7fc700 1 -- 172.21.15.89:0/105051811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c1e6140 con 0x7f552c0ca4a0 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f552a7fc700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552c0ca4a0 0x7f552c0ca910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f551c00c960 tx=0x7f551c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f5513fff700 1 -- 172.21.15.89:0/105051811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c004870 con 0x7f552c0ca4a0 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f5513fff700 1 -- 172.21.15.89:0/105051811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c0049f0 con 0x7f552c0ca4a0 2023-07-06T02:23:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c1e6420 con 0x7f552c0ca4a0 2023-07-06T02:23:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f5513fff700 1 -- 172.21.15.89:0/105051811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c004da0 con 0x7f552c0ca4a0 2023-07-06T02:23:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.810+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c1e5a90 con 0x7f552c0ca4a0 2023-07-06T02:23:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.811+0000 7f5513fff700 1 -- 172.21.15.89:0/105051811 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f551c0070d0 con 0x7f552c0ca4a0 2023-07-06T02:23:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.812+0000 7f5513fff700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f551806ace0 0x7f551806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.812+0000 7f5513fff700 1 -- 172.21.15.89:0/105051811 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f551c08af50 con 0x7f552c0ca4a0 2023-07-06T02:23:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.812+0000 7f552affd700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f551806ace0 0x7f551806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.812+0000 7f552affd700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f551806ace0 0x7f551806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5520009a60 tx=0x7f5520008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 Processor -- start 2023-07-06T02:23:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 -- start start 2023-07-06T02:23:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da73730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378d94dc60 con 0x56378da46830 2023-07-06T02:23:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378d9a09f0 con 0x56378da70470 2023-07-06T02:23:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.814+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56378d999e80 con 0x56378da70cd0 2023-07-06T02:23:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33230/0 (socket says 172.21.15.89:33230) 2023-07-06T02:23:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 -- 172.21.15.89:0/3867235483 learned_addr learned my addr 172.21.15.89:0/3867235483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552affd700 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552b7fe700 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da73730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 -- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 msgr2=0x56378da73730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da73730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 -- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 msgr2=0x56378da8f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 -- 172.21.15.89:0/3867235483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56378d7c7d90 con 0x56378da46830 2023-07-06T02:23:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f552a7fc700 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da731f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f551c015040 tx=0x7f551c0593c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9562f65252a9d020 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f55117fa700 1 -- 172.21.15.89:0/3867235483 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c061db0 con 0x56378da46830 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f55117fa700 1 -- 172.21.15.89:0/3867235483 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c071170 con 0x56378da46830 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.815+0000 7f55117fa700 1 -- 172.21.15.89:0/3867235483 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c056170 con 0x56378da46830 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 -- 172.21.15.89:0/3867235483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 msgr2=0x56378da731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da731f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f551c015040 tx=0x7f551c0593c0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 -- 172.21.15.89:0/3867235483 shutdown_connections 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da73730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 --2- 172.21.15.89:0/3867235483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da731f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 -- 172.21.15.89:0/3867235483 >> 172.21.15.89:0/3867235483 conn(0x56378da3de80 msgr2=0x56378da8e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 -- 172.21.15.89:0/3867235483 shutdown_connections 2023-07-06T02:23:24.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.816+0000 7f553daa2b80 1 -- 172.21.15.89:0/3867235483 wait complete. 2023-07-06T02:23:24.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 -- start start 2023-07-06T02:23:24.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 Processor -- start 2023-07-06T02:23:24.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378daa01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da48100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378d939a70 con 0x56378da46830 2023-07-06T02:23:24.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378d939770 con 0x56378da70470 2023-07-06T02:23:24.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56378d81d8d0 con 0x56378da70cd0 2023-07-06T02:23:24.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f553daa2b80 10 client.0 _ll_register_callbacks cb 0x56378d937710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T02:23:24.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f552affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378daa01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.817+0000 7f552affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378daa01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33234/0 (socket says 172.21.15.89:33234) 2023-07-06T02:23:24.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552affd700 1 -- 172.21.15.89:0/244509606 learned_addr learned my addr 172.21.15.89:0/244509606 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:24.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552b7fe700 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da48100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552a7fc700 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552affd700 1 -- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 msgr2=0x56378da48100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552affd700 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da48100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552affd700 1 -- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 msgr2=0x56378da8cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552affd700 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552affd700 1 -- 172.21.15.89:0/244509606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c1e5a90 con 0x56378da46830 2023-07-06T02:23:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f552affd700 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378daa01c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5520016a10 tx=0x7f55200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fb53be0a82d65c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f550a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-07-06T02:23:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f550a7fc700 1 -- 172.21.15.89:0/244509606 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520014e70 con 0x56378da46830 2023-07-06T02:23:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f550a7fc700 1 -- 172.21.15.89:0/244509606 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5520023850 con 0x56378da46830 2023-07-06T02:23:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56378da3b490 con 0x56378da46830 2023-07-06T02:23:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f550a7fc700 1 -- 172.21.15.89:0/244509606 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5520023a10 con 0x56378da46830 2023-07-06T02:23:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f553daa2b80 10 client.4917 fetch_fsmap learned FSMap version 8 2023-07-06T02:23:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56378d7b0c10 con 0x56378da46830 2023-07-06T02:23:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.818+0000 7f550a7fc700 1 -- 172.21.15.89:0/244509606 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56378da3b490 con 0x56378da46830 2023-07-06T02:23:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f550a7fc700 1 -- 172.21.15.89:0/244509606 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f5520023b70 con 0x56378da46830 2023-07-06T02:23:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f553daa2b80 10 client.4917 fetch_fsmap finished waiting for FSMap version 8 2023-07-06T02:23:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f553daa2b80 10 client.4917 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T02:23:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f553daa2b80 20 client.4917 populate_metadata read hostname 'smithi089' 2023-07-06T02:23:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f553daa2b80 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da901c0 0x56378da90590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f553daa2b80 4 client.4917 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T02:23:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 --> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56378d992e60 con 0x56378da901c0 2023-07-06T02:23:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f552a7fc700 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da901c0 0x56378da90590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f550a7fc700 10 client.4917 ms_handle_connect on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f552a7fc700 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da901c0 0x56378da90590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9cf5505ee74053b server_cookie=f87343764cdaf4bd in_seq=0 out_seq=0 2023-07-06T02:23:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f550a7fc700 1 -- 172.21.15.89:0/244509606 <== mds.0 v2:172.21.15.89:6836/1524050092 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56378d992e60 con 0x56378da901c0 2023-07-06T02:23:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f550a7fc700 10 client.4917 handle_command_reply: tid=0 2023-07-06T02:23:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.819+0000 7f553daa2b80 1 client.4917 shutdown 2023-07-06T02:23:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 msgr2=0x56378daa01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378daa01c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5520016a10 tx=0x7f55200047c0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 shutdown_connections 2023-07-06T02:23:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da901c0 0x56378da90590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da48100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f550a7fc700 0 client.4917 ms_handle_reset on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378daa01c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 --2- 172.21.15.89:0/244509606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da8cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 >> 172.21.15.89:0/244509606 conn(0x56378da3de80 msgr2=0x56378da6dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 shutdown_connections 2023-07-06T02:23:24.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 1 -- 172.21.15.89:0/244509606 wait complete. 2023-07-06T02:23:24.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.820+0000 7f553daa2b80 20 client.4917 trim_cache size 0 max 16384 2023-07-06T02:23:24.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 Processor -- start 2023-07-06T02:23:24.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 -- start start 2023-07-06T02:23:24.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da6dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70470 0x56378da6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70cd0 0x56378da8a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378da777a0 con 0x56378da46830 2023-07-06T02:23:24.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378da44630 con 0x56378da70cd0 2023-07-06T02:23:24.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.838+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56378da44ad0 con 0x56378da70470 2023-07-06T02:23:24.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da6dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da6dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33244/0 (socket says 172.21.15.89:33244) 2023-07-06T02:23:24.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 -- 172.21.15.89:0/2473093010 learned_addr learned my addr 172.21.15.89:0/2473093010 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:24.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 -- 172.21.15.89:0/2473093010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70470 msgr2=0x56378da6e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 --2- 172.21.15.89:0/2473093010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70470 0x56378da6e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 -- 172.21.15.89:0/2473093010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70cd0 msgr2=0x56378da8a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 --2- 172.21.15.89:0/2473093010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70cd0 0x56378da8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 -- 172.21.15.89:0/2473093010 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56378d7ee200 con 0x56378da46830 2023-07-06T02:23:24.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f552affd700 1 --2- 172.21.15.89:0/2473093010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da6dae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5520009fd0 tx=0x7f5520026070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7593374151d63e48 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f5511ffb700 1 -- 172.21.15.89:0/2473093010 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200238a0 con 0x56378da46830 2023-07-06T02:23:24.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f5511ffb700 1 -- 172.21.15.89:0/2473093010 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520023a00 con 0x56378da46830 2023-07-06T02:23:24.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.839+0000 7f5511ffb700 1 -- 172.21.15.89:0/2473093010 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520014840 con 0x56378da46830 2023-07-06T02:23:24.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 -- 172.21.15.89:0/2473093010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 msgr2=0x56378da6dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 --2- 172.21.15.89:0/2473093010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da6dae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5520009fd0 tx=0x7f5520026070 comp rx=0 tx=0).stop 2023-07-06T02:23:24.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 -- 172.21.15.89:0/2473093010 shutdown_connections 2023-07-06T02:23:24.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 --2- 172.21.15.89:0/2473093010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70cd0 0x56378da8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 --2- 172.21.15.89:0/2473093010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378da6dae0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 --2- 172.21.15.89:0/2473093010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70470 0x56378da6e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 -- 172.21.15.89:0/2473093010 >> 172.21.15.89:0/2473093010 conn(0x56378da3de80 msgr2=0x56378da492c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 -- 172.21.15.89:0/2473093010 shutdown_connections 2023-07-06T02:23:24.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 -- 172.21.15.89:0/2473093010 wait complete. 2023-07-06T02:23:24.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 -- start start 2023-07-06T02:23:24.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.840+0000 7f553daa2b80 1 Processor -- start 2023-07-06T02:23:24.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378dade5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da6dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f553daa2b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da6e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f553daa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378da8ac70 con 0x56378da46830 2023-07-06T02:23:24.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378da6f160 con 0x56378da70470 2023-07-06T02:23:24.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f553daa2b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56378da42110 con 0x56378da70cd0 2023-07-06T02:23:24.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f553daa2b80 10 client.0 _ll_register_callbacks cb 0x56378d937710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T02:23:24.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378dade5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378dade5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33248/0 (socket says 172.21.15.89:33248) 2023-07-06T02:23:24.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 -- 172.21.15.89:0/655529479 learned_addr learned my addr 172.21.15.89:0/655529479 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:24.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552a7fc700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da6dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552b7fe700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da6e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 -- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 msgr2=0x56378da6e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da6e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 -- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 msgr2=0x56378da6dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:24.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da6dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552affd700 1 -- 172.21.15.89:0/655529479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56378d7b0c10 con 0x56378da46830 2023-07-06T02:23:24.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.841+0000 7f552b7fe700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da6e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:24.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f552affd700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378dade5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5520023fe0 tx=0x7f5520019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d69951167ce6190 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:24.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f550a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-07-06T02:23:24.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f550a7fc700 1 -- 172.21.15.89:0/655529479 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520005e30 con 0x56378da46830 2023-07-06T02:23:24.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f550a7fc700 1 -- 172.21.15.89:0/655529479 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f552002eb70 con 0x56378da46830 2023-07-06T02:23:24.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f551c071170 con 0x56378da46830 2023-07-06T02:23:24.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f550a7fc700 1 -- 172.21.15.89:0/655529479 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5520023cb0 con 0x56378da46830 2023-07-06T02:23:24.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 10 client.4923 fetch_fsmap learned FSMap version 8 2023-07-06T02:23:24.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56378d7c6260 con 0x56378da46830 2023-07-06T02:23:24.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f550a7fc700 1 -- 172.21.15.89:0/655529479 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520050020 con 0x56378da46830 2023-07-06T02:23:24.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f550a7fc700 1 -- 172.21.15.89:0/655529479 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f552002ed30 con 0x56378da46830 2023-07-06T02:23:24.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 10 client.4923 fetch_fsmap finished waiting for FSMap version 8 2023-07-06T02:23:24.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 10 client.4923 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T02:23:24.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 20 client.4923 populate_metadata read hostname 'smithi089' 2023-07-06T02:23:24.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da42880 0x56378da42c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 4 client.4923 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-07-06T02:23:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 --> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56378d955000 con 0x56378da42880 2023-07-06T02:23:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.842+0000 7f552a7fc700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da42880 0x56378da42c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f550a7fc700 10 client.4923 ms_handle_connect on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f552a7fc700 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da42880 0x56378da42c50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613c910999bb753b server_cookie=46a7b04c3bb22fb in_seq=0 out_seq=0 2023-07-06T02:23:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f550a7fc700 1 -- 172.21.15.89:0/655529479 <== mds.0 v2:172.21.15.89:6836/1524050092 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56378d955000 con 0x56378da42880 2023-07-06T02:23:24.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f550a7fc700 10 client.4923 handle_command_reply: tid=0 2023-07-06T02:23:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 client.4923 shutdown 2023-07-06T02:23:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 msgr2=0x56378dade5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378dade5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5520023fe0 tx=0x7f5520019000 comp rx=0 tx=0).stop 2023-07-06T02:23:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 shutdown_connections 2023-07-06T02:23:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:6836/1524050092,v1:172.21.15.89:6837/1524050092] conn(0x56378da42880 0x56378da42c50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56378da70cd0 0x56378da6e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f550a7fc700 0 client.4923 ms_handle_reset on v2:172.21.15.89:6836/1524050092 2023-07-06T02:23:24.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56378da46830 0x56378dade5d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 --2- 172.21.15.89:0/655529479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56378da70470 0x56378da6dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 >> 172.21.15.89:0/655529479 conn(0x56378da3de80 msgr2=0x56378da43600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 shutdown_connections 2023-07-06T02:23:24.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 1 -- 172.21.15.89:0/655529479 wait complete. 2023-07-06T02:23:24.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.843+0000 7f553daa2b80 20 client.4923 trim_cache size 0 max 16384 2023-07-06T02:23:24.858 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-07-06T02:23:24.858 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "b61ed025-dcde-49f5-a87d-8904066cfa54", 2023-07-06T02:23:24.858 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-07-06T02:23:24.858 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4299, 2023-07-06T02:23:24.858 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-07-06T02:23:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-07-06T02:23:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-07-06T02:23:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 31.442649337999999, 2023-07-06T02:23:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 8, 2023-07-06T02:23:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 24, 2023-07-06T02:23:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 24, 2023-07-06T02:23:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 37.402586567999997 2023-07-06T02:23:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-07-06T02:23:24.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f551806ace0 msgr2=0x7f551806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f551806ace0 0x7f551806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5520009a60 tx=0x7f5520008040 comp rx=0 tx=0).stop 2023-07-06T02:23:24.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552c0ca4a0 msgr2=0x7f552c0ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:24.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552c0ca4a0 0x7f552c0ca910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f551c00c960 tx=0x7f551c000fe0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 shutdown_connections 2023-07-06T02:23:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c1e50e0 0x7f552c1e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5514004430 0x7f552c0c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f551806ace0 0x7f551806d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 --2- 172.21.15.89:0/105051811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f552c0ca4a0 0x7f552c0ca910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 >> 172.21.15.89:0/105051811 conn(0x7f552c0cc490 msgr2=0x7f552c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 shutdown_connections 2023-07-06T02:23:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:24.844+0000 7f553191d700 1 -- 172.21.15.89:0/105051811 wait complete. 2023-07-06T02:23:24.863 INFO:teuthology.run_tasks:Running task kclient... 2023-07-06T02:23:24.877 INFO:tasks.kclient:Mounting kernel clients... 2023-07-06T02:23:24.877 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-06T02:23:24.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:24.878 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-07-06T02:23:24.894 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-07-06T02:23:24.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:24.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-06T02:23:24.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:24.973 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-07-06T02:23:25.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.072 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-07-06T02:23:25.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-07-06T02:23:25.159 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2023-07-06T02:23:25.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:23:25.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.161 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-07-06T02:23:25.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.182 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-07-06T02:23:25.257 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2023-07-06T02:23:25.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:23:25.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.259 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-07-06T02:23:25.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.318 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-07-06T02:23:25.389 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2023-07-06T02:23:25.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:23:25.391 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-07-06T02:23:25.391 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T02:23:25.391 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T02:23:25.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T02:23:25.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b52e0d700 1 Processor -- start 2023-07-06T02:23:25.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b52e0d700 1 -- start start 2023-07-06T02:23:25.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b52e0d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3b4c0b1490 0x7f3b4c0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:25.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b52e0d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:25.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b52e0d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c0cab50 con 0x7f3b4c15c690 2023-07-06T02:23:25.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b52e0d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4c15de50 con 0x7f3b4c0b1490 2023-07-06T02:23:25.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b52e0d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0b1e50 con 0x7f3b4c06bfa0 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.512+0000 7f3b4bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3b4c0b1490 0x7f3b4c0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3b4c0b1490 0x7f3b4c0b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33258/0 (socket says 172.21.15.89:33258) 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4bfff700 1 -- 172.21.15.89:0/173283923 learned_addr learned my addr 172.21.15.89:0/173283923 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b513aa700 1 --2- 172.21.15.89:0/173283923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4bfff700 1 -- 172.21.15.89:0/173283923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 msgr2=0x7f3b4c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4bfff700 1 --2- 172.21.15.89:0/173283923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4bfff700 1 -- 172.21.15.89:0/173283923 >> v1:172.21.15.172:6789/0 conn(0x7f3b4c15c690 legacy=0x7f3b4c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:25.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4bfff700 1 -- 172.21.15.89:0/173283923 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c06c920 con 0x7f3b4c0b1490 2023-07-06T02:23:25.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4bfff700 1 --2- 172.21.15.89:0/173283923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c0b1910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098f0 tx=0x7f3b34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d07e3da4de308de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:25.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4b7fe700 1 -- 172.21.15.89:0/173283923 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400b8a0 con 0x7f3b4c0b1490 2023-07-06T02:23:25.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4b7fe700 1 -- 172.21.15.89:0/173283923 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3400ba00 con 0x7f3b4c0b1490 2023-07-06T02:23:25.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.513+0000 7f3b4b7fe700 1 -- 172.21.15.89:0/173283923 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400bd90 con 0x7f3b4c0b1490 2023-07-06T02:23:25.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 -- 172.21.15.89:0/173283923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 msgr2=0x7f3b4c0b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:25.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/173283923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c0b1910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098f0 tx=0x7f3b34009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 -- 172.21.15.89:0/173283923 shutdown_connections 2023-07-06T02:23:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/173283923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/173283923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c0b1910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 -- 172.21.15.89:0/173283923 >> 172.21.15.89:0/173283923 conn(0x7f3b4c0cc490 msgr2=0x7f3b4c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 -- 172.21.15.89:0/173283923 shutdown_connections 2023-07-06T02:23:25.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.514+0000 7f3b52e0d700 1 -- 172.21.15.89:0/173283923 wait complete. 2023-07-06T02:23:25.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.515+0000 7f3b52e0d700 1 Processor -- start 2023-07-06T02:23:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.515+0000 7f3b52e0d700 1 -- start start 2023-07-06T02:23:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b52e0d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b52e0d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b52e0d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4c15c690 0x7f3b4c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:25.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b52e0d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c15d230 con 0x7f3b4c0b1490 2023-07-06T02:23:25.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b52e0d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c15c090 con 0x7f3b4c15c690 2023-07-06T02:23:25.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b52e0d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c159d30 con 0x7f3b4c06bfa0 2023-07-06T02:23:25.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:25.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33266/0 (socket says 172.21.15.89:33266) 2023-07-06T02:23:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 -- 172.21.15.89:0/3536401224 learned_addr learned my addr 172.21.15.89:0/3536401224 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b513aa700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4c15c690 0x7f3b4c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b50ba9700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 -- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 msgr2=0x7f3b4c0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c0b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 -- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4c15c690 msgr2=0x7f3b4c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4c15c690 0x7f3b4c1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 -- 172.21.15.89:0/3536401224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b34009550 con 0x7f3b4c0b1490 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.516+0000 7f3b4bfff700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c1592b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b34000c00 tx=0x7f3b34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.517+0000 7f3b497fa700 1 -- 172.21.15.89:0/3536401224 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340036a0 con 0x7f3b4c0b1490 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.517+0000 7f3b497fa700 1 -- 172.21.15.89:0/3536401224 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b34003800 con 0x7f3b4c0b1490 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.517+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c159fc0 con 0x7f3b4c0b1490 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.517+0000 7f3b497fa700 1 -- 172.21.15.89:0/3536401224 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34003b90 con 0x7f3b4c0b1490 2023-07-06T02:23:25.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.517+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c15d410 con 0x7f3b4c0b1490 2023-07-06T02:23:25.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.518+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c066fb0 con 0x7f3b4c0b1490 2023-07-06T02:23:25.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.518+0000 7f3b497fa700 1 -- 172.21.15.89:0/3536401224 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b34003cf0 con 0x7f3b4c0b1490 2023-07-06T02:23:25.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.518+0000 7f3b497fa700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3b2c06ace0 0x7f3b2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:25.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.518+0000 7f3b497fa700 1 -- 172.21.15.89:0/3536401224 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3b3408c840 con 0x7f3b4c0b1490 2023-07-06T02:23:25.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.518+0000 7f3b50ba9700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3b2c06ace0 0x7f3b2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:25.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.519+0000 7f3b50ba9700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3b2c06ace0 0x7f3b2c06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0b0440 tx=0x7f3b44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.523+0000 7f3b497fa700 1 -- 172.21.15.89:0/3536401224 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3b34057690 con 0x7f3b4c0b1490 2023-07-06T02:23:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.754+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3b4c0cbf90 con 0x7f3b4c0b1490 2023-07-06T02:23:25.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.754+0000 7f3b497fa700 1 -- 172.21.15.89:0/3536401224 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3b3405acb0 con 0x7f3b4c0b1490 2023-07-06T02:23:25.758 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-06T02:23:25.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3b2c06ace0 msgr2=0x7f3b2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:25.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3b2c06ace0 0x7f3b2c06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0b0440 tx=0x7f3b44009040 comp rx=0 tx=0).stop 2023-07-06T02:23:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 msgr2=0x7f3b4c1592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c1592b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b34000c00 tx=0x7f3b34004750 comp rx=0 tx=0).stop 2023-07-06T02:23:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 shutdown_connections 2023-07-06T02:23:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4c15c690 0x7f3b4c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b4c06bfa0 0x7f3b4c0b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3b2c06ace0 0x7f3b2c06d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 --2- 172.21.15.89:0/3536401224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b4c0b1490 0x7f3b4c1592b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 >> 172.21.15.89:0/3536401224 conn(0x7f3b4c0cc490 msgr2=0x7f3b4c15aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 shutdown_connections 2023-07-06T02:23:25.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:25.757+0000 7f3b52e0d700 1 -- 172.21.15.89:0/3536401224 wait complete. 2023-07-06T02:23:25.772 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T02:23:25.773 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T02:23:25.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-07-06T02:23:25.773 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T02:23:25.774 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T02:23:25.774 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T02:23:25.774 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T02:23:25.775 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T02:23:25.775 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T02:23:25.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.776 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-07-06T02:23:25.794 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:25.794 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:25.794 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:25.794 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:25.795 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:25.795 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:25.796 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:25.796 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:25.796 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T02:23:25.796 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T02:23:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42034sec preferred_lft 42034sec 2023-07-06T02:23:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5513:ae8f:6526:fe98/64 scope link noprefixroute 2023-07-06T02:23:25.798 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:25.798 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T02:23:25.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.799 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:25.799 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:25.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2023-07-06T02:23:25.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2023-07-06T02:23:25.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2023-07-06T02:23:25.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T02:23:25.800 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:25.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.911 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T02:23:25.937 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-07-06T02:23:25.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.938 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-07-06T02:23:25.998 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-07-06T02:23:25.998 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T02:23:25.998 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T02:23:25.998 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T02:23:25.998 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T02:23:25.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:25.999 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:25.999 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:26.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T02:23:26.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T02:23:26.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T02:23:26.000 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:26.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:26.102 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-07-06T02:23:26.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:26.119 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-07-06T02:23:26.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:26.175 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:26.176 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:26.176 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T02:23:26.176 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-06T02:23:26.177 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:26.269 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-06T02:23:26.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:26.270 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:26.270 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:26.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-06T02:23:26.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T02:23:26.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-06T02:23:26.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-06T02:23:26.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-06T02:23:26.272 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:26.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:26.503 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:26.504 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:26.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-07-06T02:23:26.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T02:23:26.504 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:26.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:26.629 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T02:23:26.648 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T02:23:26.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:26.649 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T02:23:26.704 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-06T02:23:26.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:26.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:23:26.790 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:23:26.790 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-06T02:23:26.790 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-06T02:23:26.925 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:26.926 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:27.000 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:23:27.000 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-06T02:23:27.000 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:23:27.000 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:27.000 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:27.001 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-07-06T02:23:27.001 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-07-06T02:23:27.001 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=b61ed025-dcde-49f5-a87d-8904066cfa54" will pass to kernel 2023-07-06T02:23:27.002 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T02:23:27.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.003 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T02:23:27.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.029 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:27.101 INFO:teuthology.orchestra.run.smithi089.stdout:client4601 2023-07-06T02:23:27.101 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:27.101 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:27.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.158 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:27.229 INFO:teuthology.orchestra.run.smithi089.stdout:client4601 2023-07-06T02:23:27.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.230 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:27.300 INFO:teuthology.orchestra.run.smithi089.stdout:client4601 2023-07-06T02:23:27.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.301 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b61ed025-dcde-49f5-a87d-8904066cfa54.client4601/status') 2023-07-06T02:23:27.381 INFO:teuthology.orchestra.run.smithi089.stdout:instance: client.4601 (3)172.21.15.89:0/3890354368 2023-07-06T02:23:27.381 INFO:teuthology.orchestra.run.smithi089.stdout:blocklisted: false 2023-07-06T02:23:27.381 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records in 2023-07-06T02:23:27.382 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records out 2023-07-06T02:23:27.382 INFO:teuthology.orchestra.run.smithi089.stderr:70 bytes copied, 2.1746e-05 s, 3.2 MB/s 2023-07-06T02:23:27.383 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-07-06T02:23:27.383 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T02:23:27.383 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T02:23:27.384 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T02:23:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.541+0000 7f23d6012700 1 Processor -- start 2023-07-06T02:23:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.541+0000 7f23d6012700 1 -- start start 2023-07-06T02:23:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23d6012700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f23d0070f20 0x7f23d00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23d6012700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23d6012700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00c9fa0 con 0x7f23d006e790 2023-07-06T02:23:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23d6012700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d006f290 con 0x7f23d0070f20 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23d6012700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d006ecd0 con 0x7f23d00712f0 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23ceffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23cf7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f23d0070f20 0x7f23d00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23cffff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f23d006e790 0x7f23d006b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:33936/0 (socket says 172.21.15.89:33936) 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23cffff700 1 -- 172.21.15.89:0/492028441 learned_addr learned my addr 172.21.15.89:0/492028441 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23ceffd700 1 --2- 172.21.15.89:0/492028441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d006e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:56484/0 (socket says 172.21.15.89:56484) 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23ceffd700 1 -- 172.21.15.89:0/492028441 >> v2:172.21.15.172:3300/0 conn(0x7f23d0070f20 msgr2=0x7f23d00c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:27.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23ceffd700 1 --2- 172.21.15.89:0/492028441 >> v2:172.21.15.172:3300/0 conn(0x7f23d0070f20 0x7f23d00c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23ceffd700 1 -- 172.21.15.89:0/492028441 >> v1:172.21.15.172:6789/0 conn(0x7f23d006e790 legacy=0x7f23d006b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:27.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23ceffd700 1 -- 172.21.15.89:0/492028441 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d006efb0 con 0x7f23d00712f0 2023-07-06T02:23:27.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23ce7fc700 1 -- 172.21.15.89:0/492028441 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772947141 0 0) 0x7f23d00c9fa0 con 0x7f23d006e790 2023-07-06T02:23:27.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.542+0000 7f23cf7fe700 1 --2- 172.21.15.89:0/492028441 >> v2:172.21.15.172:3300/0 conn(0x7f23d0070f20 0x7f23d00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:27.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23ceffd700 1 --2- 172.21.15.89:0/492028441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d006e250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009840 tx=0x7f23c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a82b719adb0ffb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:27.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23ce7fc700 1 -- 172.21.15.89:0/492028441 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4011830 con 0x7f23d00712f0 2023-07-06T02:23:27.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23ce7fc700 1 -- 172.21.15.89:0/492028441 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c4011990 con 0x7f23d00712f0 2023-07-06T02:23:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23ce7fc700 1 -- 172.21.15.89:0/492028441 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4011d20 con 0x7f23d00712f0 2023-07-06T02:23:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 -- 172.21.15.89:0/492028441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 msgr2=0x7f23d006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 --2- 172.21.15.89:0/492028441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d006e250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009840 tx=0x7f23c4009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 -- 172.21.15.89:0/492028441 shutdown_connections 2023-07-06T02:23:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 --2- 172.21.15.89:0/492028441 >> v2:172.21.15.172:3300/0 conn(0x7f23d0070f20 0x7f23d00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 --2- 172.21.15.89:0/492028441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d006e250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 -- 172.21.15.89:0/492028441 >> 172.21.15.89:0/492028441 conn(0x7f23d00cf1c0 msgr2=0x7f23d00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 -- 172.21.15.89:0/492028441 shutdown_connections 2023-07-06T02:23:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.543+0000 7f23d6012700 1 -- 172.21.15.89:0/492028441 wait complete. 2023-07-06T02:23:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.545+0000 7f23d6012700 1 Processor -- start 2023-07-06T02:23:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.545+0000 7f23d6012700 1 -- start start 2023-07-06T02:23:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23d6012700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d006e790 0x7f23d01552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23d6012700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23d0070f20 0x7f23d00aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23d6012700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d01f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23d6012700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00cb650 con 0x7f23d0070f20 2023-07-06T02:23:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23d6012700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d014b140 con 0x7f23d006e790 2023-07-06T02:23:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23d6012700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d014b420 con 0x7f23d00712f0 2023-07-06T02:23:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23d0070f20 0x7f23d00aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23d0070f20 0x7f23d00aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33284/0 (socket says 172.21.15.89:33284) 2023-07-06T02:23:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 -- 172.21.15.89:0/92734942 learned_addr learned my addr 172.21.15.89:0/92734942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23cf7fe700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d006e790 0x7f23d01552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23cffff700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d01f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 -- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 msgr2=0x7f23d01f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:27.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d01f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 -- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d006e790 msgr2=0x7f23d01552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:27.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d006e790 0x7f23d01552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23ceffd700 1 -- 172.21.15.89:0/92734942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c00094a0 con 0x7f23d0070f20 2023-07-06T02:23:27.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.546+0000 7f23cf7fe700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d006e790 0x7f23d01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:27.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.547+0000 7f23ceffd700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23d0070f20 0x7f23d00aed90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23c4014040 tx=0x7f23c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:27.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.547+0000 7f23b7fff700 1 -- 172.21.15.89:0/92734942 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40036a0 con 0x7f23d0070f20 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.547+0000 7f23b7fff700 1 -- 172.21.15.89:0/92734942 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c401f070 con 0x7f23d0070f20 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.547+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c40094a0 con 0x7f23d0070f20 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.547+0000 7f23b7fff700 1 -- 172.21.15.89:0/92734942 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4003a00 con 0x7f23d0070f20 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.547+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d00ca620 con 0x7f23d0070f20 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.548+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d00676c0 con 0x7f23d0070f20 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.548+0000 7f23b7fff700 1 -- 172.21.15.89:0/92734942 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23c4003b60 con 0x7f23d0070f20 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.548+0000 7f23b7fff700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f23bc06ace0 0x7f23bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:27.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.549+0000 7f23b7fff700 1 -- 172.21.15.89:0/92734942 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f23c4021070 con 0x7f23d0070f20 2023-07-06T02:23:27.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.549+0000 7f23cf7fe700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f23bc06ace0 0x7f23bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:27.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.549+0000 7f23cf7fe700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f23bc06ace0 0x7f23bc06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009dc0 tx=0x7f23c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:27.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.553+0000 7f23b7fff700 1 -- 172.21.15.89:0/92734942 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23c4055df0 con 0x7f23d0070f20 2023-07-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.784+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23d0074270 con 0x7f23d0070f20 2023-07-06T02:23:27.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.784+0000 7f23b7fff700 1 -- 172.21.15.89:0/92734942 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f23c4015070 con 0x7f23d0070f20 2023-07-06T02:23:27.788 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-06T02:23:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f23bc06ace0 msgr2=0x7f23bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f23bc06ace0 0x7f23bc06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009dc0 tx=0x7f23c0008040 comp rx=0 tx=0).stop 2023-07-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23d0070f20 msgr2=0x7f23d00aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23d0070f20 0x7f23d00aed90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23c4014040 tx=0x7f23c4004750 comp rx=0 tx=0).stop 2023-07-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 shutdown_connections 2023-07-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00712f0 0x7f23d01f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d006e790 0x7f23d01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f23bc06ace0 0x7f23bc06d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 --2- 172.21.15.89:0/92734942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23d0070f20 0x7f23d00aed90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 >> 172.21.15.89:0/92734942 conn(0x7f23d00cf1c0 msgr2=0x7f23d0157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 shutdown_connections 2023-07-06T02:23:27.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:27.786+0000 7f23d6012700 1 -- 172.21.15.89:0/92734942 wait complete. 2023-07-06T02:23:27.801 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T02:23:27.801 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T02:23:27.801 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-07-06T02:23:27.802 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-06T02:23:27.802 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T02:23:27.802 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T02:23:27.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T02:23:27.803 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T02:23:27.803 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-06T02:23:27.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.803 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-07-06T02:23:27.820 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:27.821 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:27.821 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:27.821 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:27.822 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T02:23:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T02:23:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42032sec preferred_lft 42032sec 2023-07-06T02:23:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5513:ae8f:6526:fe98/64 scope link noprefixroute 2023-07-06T02:23:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 7a:a2:e5:23:3b:bc brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5c4c:ff:fec3:ff8a/64 scope link 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T02:23:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 7a:a2:e5:23:3b:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T02:23:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::78a2:e5ff:fe23:3bbc/64 scope link tentative 2023-07-06T02:23:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:27.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.824 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T02:23:27.884 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-07-06T02:23:27.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.886 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-07-06T02:23:27.943 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-07-06T02:23:27.944 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T02:23:27.944 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T02:23:27.944 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T02:23:27.944 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T02:23:27.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:27.945 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:27.945 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:27.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T02:23:27.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T02:23:27.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T02:23:27.946 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:28.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.048 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-07-06T02:23:28.065 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T02:23:28.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.066 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-07-06T02:23:28.122 INFO:teuthology.orchestra.run.smithi089.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-06T02:23:28.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.123 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:28.123 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:28.124 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T02:23:28.124 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-06T02:23:28.124 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:28.188 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:23:28.206 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:23:28.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.216 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T02:23:28.329 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.330 INFO:teuthology.orchestra.run.smithi089.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T02:23:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 26:36:0a:f7:b8:86 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T02:23:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T02:23:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::2436:aff:fef7:b886/64 scope link 2023-07-06T02:23:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.341 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T02:23:28.421 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.422 INFO:teuthology.orchestra.run.smithi089.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T02:23:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 26:36:0a:f7:b8:86 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T02:23:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T02:23:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::2436:aff:fef7:b886/64 scope link 2023-07-06T02:23:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:28.438 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-06T02:23:28.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.439 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:28.439 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:28.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-06T02:23:28.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T02:23:28.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-06T02:23:28.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-06T02:23:28.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-06T02:23:28.440 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:28.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.684 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:23:28.685 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:23:28.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.1 up 2023-07-06T02:23:28.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T02:23:28.685 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:23:28.800 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:28.801 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-06T02:23:28.819 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-06T02:23:28.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:28.819 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-06T02:23:28.875 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-06T02:23:28.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:28.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-06T02:23:28.960 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-06T02:23:28.960 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-07-06T02:23:28.961 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:23:28.969 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-06T02:23:28.969 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-07-06T02:23:28.969 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:23:28.969 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:28.969 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:28.999 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-06T02:23:28.999 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-07-06T02:23:28.999 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:23:29.000 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:29.000 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:29.000 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-07-06T02:23:29.000 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-07-06T02:23:29.001 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=b61ed025-dcde-49f5-a87d-8904066cfa54" will pass to kernel 2023-07-06T02:23:29.002 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T02:23:29.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:29.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-06T02:23:29.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:29.037 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:29.109 INFO:teuthology.orchestra.run.smithi089.stdout:client4732 2023-07-06T02:23:29.109 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:29.110 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:29.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:29.166 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:29.237 INFO:teuthology.orchestra.run.smithi089.stdout:client4732 2023-07-06T02:23:29.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:29.238 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:29.309 INFO:teuthology.orchestra.run.smithi089.stdout:client4732 2023-07-06T02:23:29.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:29.310 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b61ed025-dcde-49f5-a87d-8904066cfa54.client4732/status') 2023-07-06T02:23:29.389 INFO:teuthology.orchestra.run.smithi089.stdout:instance: client.4732 (3)172.21.15.89:0/3106223649 2023-07-06T02:23:29.389 INFO:teuthology.orchestra.run.smithi089.stdout:blocklisted: false 2023-07-06T02:23:29.390 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records in 2023-07-06T02:23:29.390 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records out 2023-07-06T02:23:29.390 INFO:teuthology.orchestra.run.smithi089.stderr:70 bytes copied, 2.1573e-05 s, 3.2 MB/s 2023-07-06T02:23:29.391 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-07-06T02:23:29.391 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T02:23:29.391 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T02:23:29.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T02:23:29.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.550+0000 7f1f21100700 1 Processor -- start 2023-07-06T02:23:29.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.550+0000 7f1f21100700 1 -- start start 2023-07-06T02:23:29.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f21100700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0b1490 0x7f1f1c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:29.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f21100700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f1c0cc000 0x7f1f1c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:29.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f21100700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c0b1da0 con 0x7f1f1c06b220 2023-07-06T02:23:29.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f21100700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1c06a990 con 0x7f1f1c0cc000 2023-07-06T02:23:29.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f21100700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c06ac70 con 0x7f1f1c0b1490 2023-07-06T02:23:29.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f1c0cc000 0x7f1f1c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:29.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1f1c0cc000 0x7f1f1c0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33294/0 (socket says 172.21.15.89:33294) 2023-07-06T02:23:29.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 -- 172.21.15.89:0/2829305416 learned_addr learned my addr 172.21.15.89:0/2829305416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:29.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1ad9d700 1 --2- 172.21.15.89:0/2829305416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0b1490 0x7f1f1c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:29.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 -- 172.21.15.89:0/2829305416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0b1490 msgr2=0x7f1f1c0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:29.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 --2- 172.21.15.89:0/2829305416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0b1490 0x7f1f1c0caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 -- 172.21.15.89:0/2829305416 >> v1:172.21.15.172:6789/0 conn(0x7f1f1c06b220 legacy=0x7f1f1c15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 -- 172.21.15.89:0/2829305416 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c06d170 con 0x7f1f1c0cc000 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1ad9d700 1 --2- 172.21.15.89:0/2829305416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0b1490 0x7f1f1c0caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.551+0000 7f1f1a59c700 1 --2- 172.21.15.89:0/2829305416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c0cc000 0x7f1f1c0b1860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0098f0 tx=0x7f1f0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f96de065648c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f19d9b700 1 -- 172.21.15.89:0/2829305416 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c004230 con 0x7f1f1c0cc000 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f19d9b700 1 -- 172.21.15.89:0/2829305416 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c004390 con 0x7f1f1c0cc000 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f19d9b700 1 -- 172.21.15.89:0/2829305416 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0036a0 con 0x7f1f1c0cc000 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 -- 172.21.15.89:0/2829305416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c0cc000 msgr2=0x7f1f1c0b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:29.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 --2- 172.21.15.89:0/2829305416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c0cc000 0x7f1f1c0b1860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0098f0 tx=0x7f1f0c009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:29.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 -- 172.21.15.89:0/2829305416 shutdown_connections 2023-07-06T02:23:29.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 --2- 172.21.15.89:0/2829305416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0b1490 0x7f1f1c0caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 --2- 172.21.15.89:0/2829305416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c0cc000 0x7f1f1c0b1860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 -- 172.21.15.89:0/2829305416 >> 172.21.15.89:0/2829305416 conn(0x7f1f1c0cc490 msgr2=0x7f1f1c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:29.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 -- 172.21.15.89:0/2829305416 shutdown_connections 2023-07-06T02:23:29.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.552+0000 7f1f21100700 1 -- 172.21.15.89:0/2829305416 wait complete. 2023-07-06T02:23:29.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.553+0000 7f1f21100700 1 Processor -- start 2023-07-06T02:23:29.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f21100700 1 -- start start 2023-07-06T02:23:29.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f21100700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c06b220 0x7f1f1c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:29.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f21100700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f1c0b1490 0x7f1f1c0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:29.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f21100700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0cc000 0x7f1f1c0d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:29.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f21100700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c155f70 con 0x7f1f1c06b220 2023-07-06T02:23:29.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f21100700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1e3cd0 con 0x7f1f1c0b1490 2023-07-06T02:23:29.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f21100700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1e3fb0 con 0x7f1f1c0cc000 2023-07-06T02:23:29.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f1ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c06b220 0x7f1f1c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:29.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f1ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c06b220 0x7f1f1c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33298/0 (socket says 172.21.15.89:33298) 2023-07-06T02:23:29.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.554+0000 7f1f1ad9d700 1 -- 172.21.15.89:0/2385323234 learned_addr learned my addr 172.21.15.89:0/2385323234 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1a59c700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f1c0b1490 0x7f1f1c0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1b59e700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0cc000 0x7f1f1c0d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1ad9d700 1 -- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0cc000 msgr2=0x7f1f1c0d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1ad9d700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0cc000 0x7f1f1c0d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1ad9d700 1 -- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f1c0b1490 msgr2=0x7f1f1c0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1ad9d700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f1c0b1490 0x7f1f1c0cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1ad9d700 1 -- 172.21.15.89:0/2385323234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c009550 con 0x7f1f1c06b220 2023-07-06T02:23:29.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f1ad9d700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c06b220 0x7f1f1c152e70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c750 tx=0x7f1f1000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f037fe700 1 -- 172.21.15.89:0/2385323234 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10011800 con 0x7f1f1c06b220 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f037fe700 1 -- 172.21.15.89:0/2385323234 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10011960 con 0x7f1f1c06b220 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f037fe700 1 -- 172.21.15.89:0/2385323234 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10011cd0 con 0x7f1f1c06b220 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c1e4290 con 0x7f1f1c06b220 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.555+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c1533b0 con 0x7f1f1c06b220 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.556+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c1e47b0 con 0x7f1f1c06b220 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.556+0000 7f1f037fe700 1 -- 172.21.15.89:0/2385323234 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f10021350 con 0x7f1f1c06b220 2023-07-06T02:23:29.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.557+0000 7f1f037fe700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f0806ad10 0x7f1f0806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.557+0000 7f1f037fe700 1 -- 172.21.15.89:0/2385323234 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f100897f0 con 0x7f1f1c06b220 2023-07-06T02:23:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.557+0000 7f1f1a59c700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f0806ad10 0x7f1f0806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.557+0000 7f1f1a59c700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f0806ad10 0x7f1f0806d1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009840 tx=0x7f1f0c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:29.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.561+0000 7f1f037fe700 1 -- 172.21.15.89:0/2385323234 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1f1c1e47b0 con 0x7f1f1c06b220 2023-07-06T02:23:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.793+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f1c1e47b0 con 0x7f1f1c06b220 2023-07-06T02:23:29.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.793+0000 7f1f037fe700 1 -- 172.21.15.89:0/2385323234 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1f1c1e47b0 con 0x7f1f1c06b220 2023-07-06T02:23:29.796 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-06T02:23:29.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f0806ad10 msgr2=0x7f1f0806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:29.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f0806ad10 0x7f1f0806d1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009840 tx=0x7f1f0c008030 comp rx=0 tx=0).stop 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c06b220 msgr2=0x7f1f1c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c06b220 0x7f1f1c152e70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c750 tx=0x7f1f1000cb10 comp rx=0 tx=0).stop 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 shutdown_connections 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1f1c0cc000 0x7f1f1c0d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f1f0806ad10 0x7f1f0806d1c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f1c06b220 0x7f1f1c152e70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 --2- 172.21.15.89:0/2385323234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f1c0b1490 0x7f1f1c0cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:29.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 >> 172.21.15.89:0/2385323234 conn(0x7f1f1c0cc490 msgr2=0x7f1f1c15e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:29.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 shutdown_connections 2023-07-06T02:23:29.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:29.795+0000 7f1f21100700 1 -- 172.21.15.89:0/2385323234 wait complete. 2023-07-06T02:23:29.810 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T02:23:29.810 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T02:23:29.811 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-07-06T02:23:29.811 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T02:23:29.811 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T02:23:29.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T02:23:29.811 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T02:23:29.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T02:23:29.812 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T02:23:29.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:29.812 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-07-06T02:23:29.834 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:29.834 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:29.835 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:29.835 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:29.835 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:29.835 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T02:23:29.836 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:29.837 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T02:23:29.837 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42009sec preferred_lft 42009sec 2023-07-06T02:23:29.837 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::19f5:ff12:99a:4bde/64 scope link noprefixroute 2023-07-06T02:23:29.837 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:29.837 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T02:23:29.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:29.838 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:29.838 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:29.838 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2023-07-06T02:23:29.839 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2023-07-06T02:23:29.839 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2023-07-06T02:23:29.839 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T02:23:29.839 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:30.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.103 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T02:23:30.130 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-07-06T02:23:30.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.132 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-07-06T02:23:30.213 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-07-06T02:23:30.213 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T02:23:30.213 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T02:23:30.213 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T02:23:30.214 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T02:23:30.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.214 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:30.215 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:30.215 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T02:23:30.215 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T02:23:30.216 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T02:23:30.216 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:30.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.456 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-07-06T02:23:30.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.473 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2023-07-06T02:23:30.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.530 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:30.531 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:30.531 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T02:23:30.532 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-06T02:23:30.532 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:30.623 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-06T02:23:30.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.624 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:30.624 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:30.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-06T02:23:30.625 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T02:23:30.625 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-06T02:23:30.625 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-06T02:23:30.625 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-06T02:23:30.625 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:30.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:30.818 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:30.819 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:30.819 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-07-06T02:23:30.820 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T02:23:30.820 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:30.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:30.928 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T02:23:30.949 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T02:23:30.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:30.949 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T02:23:31.005 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-06T02:23:31.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:31.006 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-06T02:23:31.100 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-06T02:23:31.100 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-07-06T02:23:31.100 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2023-07-06T02:23:31.194 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-06T02:23:31.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-07-06T02:23:31.195 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2023-07-06T02:23:31.196 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:31.196 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:31.262 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-06T02:23:31.262 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-07-06T02:23:31.263 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2023-07-06T02:23:31.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:31.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:31.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-07-06T02:23:31.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-07-06T02:23:31.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=b61ed025-dcde-49f5-a87d-8904066cfa54" will pass to kernel 2023-07-06T02:23:31.265 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T02:23:31.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:31.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T02:23:31.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:31.292 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:31.364 INFO:teuthology.orchestra.run.smithi172.stdout:client4613 2023-07-06T02:23:31.364 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:31.365 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:31.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:31.422 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:31.493 INFO:teuthology.orchestra.run.smithi172.stdout:client4613 2023-07-06T02:23:31.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:31.494 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:31.565 INFO:teuthology.orchestra.run.smithi172.stdout:client4613 2023-07-06T02:23:31.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:31.566 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b61ed025-dcde-49f5-a87d-8904066cfa54.client4613/status') 2023-07-06T02:23:31.645 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4613 (3)192.168.0.1:0/3650786142 2023-07-06T02:23:31.646 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2023-07-06T02:23:31.647 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2023-07-06T02:23:31.647 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2023-07-06T02:23:31.647 INFO:teuthology.orchestra.run.smithi172.stderr:69 bytes copied, 2.1559e-05 s, 3.2 MB/s 2023-07-06T02:23:31.647 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-07-06T02:23:31.648 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T02:23:31.648 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T02:23:31.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T02:23:31.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.780+0000 7fbad4516700 1 Processor -- start 2023-07-06T02:23:31.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.780+0000 7fbad4516700 1 -- start start 2023-07-06T02:23:31.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad4516700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:31.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad4516700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbacc160bb0 0x7fbacc0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:31.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad4516700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbacc0cb3d0 con 0x7fbacc1600b0 2023-07-06T02:23:31.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad4516700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc156e80 con 0x7fbacc160bb0 2023-07-06T02:23:31.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad4516700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0ca370 con 0x7fbacc156ab0 2023-07-06T02:23:31.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad1ab1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbacc160bb0 0x7fbacc0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:31.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad1ab1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbacc160bb0 0x7fbacc0b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33308/0 (socket says 172.21.15.89:33308) 2023-07-06T02:23:31.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad1ab1700 1 -- 172.21.15.89:0/4273022693 learned_addr learned my addr 172.21.15.89:0/4273022693 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad22b2700 1 --2- 172.21.15.89:0/4273022693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad1ab1700 1 -- 172.21.15.89:0/4273022693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 msgr2=0x7fbacc160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad1ab1700 1 --2- 172.21.15.89:0/4273022693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad1ab1700 1 -- 172.21.15.89:0/4273022693 >> v1:172.21.15.172:6789/0 conn(0x7fbacc1600b0 legacy=0x7fbacc06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.781+0000 7fbad1ab1700 1 -- 172.21.15.89:0/4273022693 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc0ca610 con 0x7fbacc160bb0 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad1ab1700 1 --2- 172.21.15.89:0/4273022693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc160bb0 0x7fbacc0b1490 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0098f0 tx=0x7fbabc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e78f550749a065 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad12b0700 1 -- 172.21.15.89:0/4273022693 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc011830 con 0x7fbacc160bb0 2023-07-06T02:23:31.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad12b0700 1 -- 172.21.15.89:0/4273022693 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc011990 con 0x7fbacc160bb0 2023-07-06T02:23:31.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad12b0700 1 -- 172.21.15.89:0/4273022693 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc011d20 con 0x7fbacc160bb0 2023-07-06T02:23:31.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 -- 172.21.15.89:0/4273022693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc160bb0 msgr2=0x7fbacc0b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:31.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 --2- 172.21.15.89:0/4273022693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc160bb0 0x7fbacc0b1490 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0098f0 tx=0x7fbabc009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:31.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 -- 172.21.15.89:0/4273022693 shutdown_connections 2023-07-06T02:23:31.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 --2- 172.21.15.89:0/4273022693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:31.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 --2- 172.21.15.89:0/4273022693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc160bb0 0x7fbacc0b1490 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:31.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 -- 172.21.15.89:0/4273022693 >> 172.21.15.89:0/4273022693 conn(0x7fbacc0cc490 msgr2=0x7fbacc0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:31.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 -- 172.21.15.89:0/4273022693 shutdown_connections 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.782+0000 7fbad4516700 1 -- 172.21.15.89:0/4273022693 wait complete. 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 Processor -- start 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 -- start start 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc1600b0 0x7fbacc155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc155f60 0x7fbacc0afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0b5830 con 0x7fbacc155f60 2023-07-06T02:23:31.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0c81a0 con 0x7fbacc1600b0 2023-07-06T02:23:31.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad4516700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0b4fb0 con 0x7fbacc156ab0 2023-07-06T02:23:31.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad2ab3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc155f60 0x7fbacc0afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:31.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad2ab3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc155f60 0x7fbacc0afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33316/0 (socket says 172.21.15.89:33316) 2023-07-06T02:23:31.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad2ab3700 1 -- 172.21.15.89:0/1435662117 learned_addr learned my addr 172.21.15.89:0/1435662117 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:31.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad22b2700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:31.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.784+0000 7fbad1ab1700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc1600b0 0x7fbacc155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:31.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad2ab3700 1 -- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 msgr2=0x7fbacc155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad2ab3700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad2ab3700 1 -- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc1600b0 msgr2=0x7fbacc155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad2ab3700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc1600b0 0x7fbacc155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad2ab3700 1 -- 172.21.15.89:0/1435662117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc009550 con 0x7fbacc155f60 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad2ab3700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc155f60 0x7fbacc0afc40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbac400cbc0 tx=0x7fbac400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbac2ffd700 1 -- 172.21.15.89:0/1435662117 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4003c00 con 0x7fbacc155f60 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbac2ffd700 1 -- 172.21.15.89:0/1435662117 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac4003d60 con 0x7fbacc155f60 2023-07-06T02:23:31.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbac2ffd700 1 -- 172.21.15.89:0/1435662117 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400aae0 con 0x7fbacc155f60 2023-07-06T02:23:31.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc1f65a0 con 0x7fbacc155f60 2023-07-06T02:23:31.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.785+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc0b5550 con 0x7fbacc155f60 2023-07-06T02:23:31.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.786+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc0c8360 con 0x7fbacc155f60 2023-07-06T02:23:31.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.786+0000 7fbac2ffd700 1 -- 172.21.15.89:0/1435662117 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbac400ac40 con 0x7fbacc155f60 2023-07-06T02:23:31.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.787+0000 7fbac2ffd700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbab006ace0 0x7fbab006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:31.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.787+0000 7fbac2ffd700 1 -- 172.21.15.89:0/1435662117 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbac40896b0 con 0x7fbacc155f60 2023-07-06T02:23:31.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.787+0000 7fbad22b2700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbab006ace0 0x7fbab006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:31.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.787+0000 7fbad22b2700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbab006ace0 0x7fbab006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b0c10 tx=0x7fbac8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:31.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:31.791+0000 7fbac2ffd700 1 -- 172.21.15.89:0/1435662117 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbac4054470 con 0x7fbacc155f60 2023-07-06T02:23:32.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.022+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbacc074270 con 0x7fbacc155f60 2023-07-06T02:23:32.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.022+0000 7fbac2ffd700 1 -- 172.21.15.89:0/1435662117 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fbac4057a90 con 0x7fbacc155f60 2023-07-06T02:23:32.026 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-06T02:23:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbab006ace0 msgr2=0x7fbab006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbab006ace0 0x7fbab006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b0c10 tx=0x7fbac8009210 comp rx=0 tx=0).stop 2023-07-06T02:23:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc155f60 msgr2=0x7fbacc0afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc155f60 0x7fbacc0afc40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbac400cbc0 tx=0x7fbac400ced0 comp rx=0 tx=0).stop 2023-07-06T02:23:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 shutdown_connections 2023-07-06T02:23:32.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbab006ace0 0x7fbab006d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:32.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbacc155f60 0x7fbacc0afc40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:32.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbacc156ab0 0x7fbacc155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:32.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 --2- 172.21.15.89:0/1435662117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc1600b0 0x7fbacc155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:32.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 >> 172.21.15.89:0/1435662117 conn(0x7fbacc0cc490 msgr2=0x7fbacc0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:32.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 shutdown_connections 2023-07-06T02:23:32.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:32.025+0000 7fbad4516700 1 -- 172.21.15.89:0/1435662117 wait complete. 2023-07-06T02:23:32.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T02:23:32.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T02:23:32.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-07-06T02:23:32.040 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-06T02:23:32.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T02:23:32.041 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T02:23:32.041 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T02:23:32.041 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T02:23:32.041 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-06T02:23:32.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.042 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-07-06T02:23:32.060 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:32.060 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:32.060 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:32.060 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.060 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:32.061 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.061 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:32.061 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:32.061 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:32.061 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:32.061 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:23:32.061 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42007sec preferred_lft 42007sec 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::19f5:ff12:99a:4bde/64 scope link noprefixroute 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T02:23:32.062 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 4a:5c:58:dc:1c:d6 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:23:32.063 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T02:23:32.063 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.063 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::4c7f:1aff:fe29:594a/64 scope link 2023-07-06T02:23:32.063 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.063 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T02:23:32.063 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 4a:5c:58:dc:1c:d6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T02:23:32.064 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::485c:58ff:fedc:1cd6/64 scope link tentative 2023-07-06T02:23:32.064 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.064 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T02:23:32.126 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-07-06T02:23:32.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.128 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-07-06T02:23:32.187 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-07-06T02:23:32.187 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T02:23:32.188 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T02:23:32.188 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T02:23:32.188 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T02:23:32.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.189 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:32.189 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:32.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T02:23:32.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T02:23:32.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T02:23:32.190 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:32.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.293 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-07-06T02:23:32.310 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T02:23:32.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.311 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2023-07-06T02:23:32.366 INFO:teuthology.orchestra.run.smithi172.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-06T02:23:32.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.367 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:32.367 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:32.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T02:23:32.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-06T02:23:32.368 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:32.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.461 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T02:23:32.494 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:32.494 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:32.495 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:32.495 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.495 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:32.495 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.495 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T02:23:32.495 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether d6:b3:7b:97:7d:4b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T02:23:32.495 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T02:23:32.496 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.496 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d4b3:7bff:fe97:7d4b/64 scope link 2023-07-06T02:23:32.496 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T02:23:32.587 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:23:32.588 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:23:32.588 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:23:32.588 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.588 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-07-06T02:23:32.588 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.588 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T02:23:32.589 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether d6:b3:7b:97:7d:4b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T02:23:32.589 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T02:23:32.589 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.589 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d4b3:7bff:fe97:7d4b/64 scope link 2023-07-06T02:23:32.589 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:23:32.611 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-06T02:23:32.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.611 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:32.612 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:32.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-06T02:23:32.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T02:23:32.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-06T02:23:32.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-06T02:23:32.613 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-06T02:23:32.613 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:32.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:32.848 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-07-06T02:23:32.848 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-07-06T02:23:32.849 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.1 up 2023-07-06T02:23:32.849 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T02:23:32.849 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-07-06T02:23:32.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:32.956 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-06T02:23:32.975 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-06T02:23:32.976 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:23:32.976 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-06T02:23:33.031 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-06T02:23:33.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:33.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-06T02:23:33.116 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-06T02:23:33.116 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-07-06T02:23:33.117 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2023-07-06T02:23:33.125 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-06T02:23:33.125 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-07-06T02:23:33.125 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2023-07-06T02:23:33.125 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:33.125 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:33.199 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-06T02:23:33.199 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-07-06T02:23:33.199 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2023-07-06T02:23:33.199 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:23:33.200 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:23:33.200 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-07-06T02:23:33.200 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-07-06T02:23:33.200 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=b61ed025-dcde-49f5-a87d-8904066cfa54" will pass to kernel 2023-07-06T02:23:33.201 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T02:23:33.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:33.201 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-06T02:23:33.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:33.228 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:33.300 INFO:teuthology.orchestra.run.smithi172.stdout:client4947 2023-07-06T02:23:33.300 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:33.301 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:33.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:33.358 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:33.430 INFO:teuthology.orchestra.run.smithi172.stdout:client4947 2023-07-06T02:23:33.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:33.431 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:23:33.503 INFO:teuthology.orchestra.run.smithi172.stdout:client4947 2023-07-06T02:23:33.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:33.504 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b61ed025-dcde-49f5-a87d-8904066cfa54.client4947/status') 2023-07-06T02:23:33.583 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4947 (3)172.21.15.172:0/1280096078 2023-07-06T02:23:33.584 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2023-07-06T02:23:33.584 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2023-07-06T02:23:33.585 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2023-07-06T02:23:33.585 INFO:teuthology.orchestra.run.smithi172.stderr:71 bytes copied, 2.5012e-05 s, 2.8 MB/s 2023-07-06T02:23:33.585 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-06T02:23:33.605 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:33.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.734+0000 7f5be7cc0700 1 Processor -- start 2023-07-06T02:23:33.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.735+0000 7f5be7cc0700 1 -- start start 2023-07-06T02:23:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.735+0000 7f5be7cc0700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5be0070f20 0x7f5be00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be7cc0700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00cb9f0 con 0x7f5be0160000 2023-07-06T02:23:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be7cc0700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be015d8e0 con 0x7f5be015c500 2023-07-06T02:23:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be7cc0700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be0071900 con 0x7f5be0070f20 2023-07-06T02:23:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be525b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5be0070f20 0x7f5be00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:33.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be625d700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5be0160000 0x7f5be0160420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58226/0 (socket says 172.21.15.89:58226) 2023-07-06T02:23:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be525b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5be0070f20 0x7f5be00713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33322/0 (socket says 172.21.15.89:33322) 2023-07-06T02:23:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be625d700 1 -- 172.21.15.89:0/3616087349 learned_addr learned my addr 172.21.15.89:0/3616087349 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be525b700 1 -- 172.21.15.89:0/3616087349 >> v1:172.21.15.172:6789/0 conn(0x7f5be015c500 legacy=0x7f5be006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be525b700 1 -- 172.21.15.89:0/3616087349 >> v1:172.21.15.89:6789/0 conn(0x7f5be0160000 legacy=0x7f5be0160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be525b700 1 -- 172.21.15.89:0/3616087349 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0160be0 con 0x7f5be0070f20 2023-07-06T02:23:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be4a5a700 1 -- 172.21.15.89:0/3616087349 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599794472 0 0) 0x7f5be00cb9f0 con 0x7f5be0160000 2023-07-06T02:23:33.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be525b700 1 --2- 172.21.15.89:0/3616087349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00713c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009840 tx=0x7f5bdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f442b958c03d417 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:33.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be4a5a700 1 -- 172.21.15.89:0/3616087349 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc011830 con 0x7f5be0070f20 2023-07-06T02:23:33.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be4a5a700 1 -- 172.21.15.89:0/3616087349 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc011990 con 0x7f5be0070f20 2023-07-06T02:23:33.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.736+0000 7f5be4a5a700 1 -- 172.21.15.89:0/3616087349 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc011d20 con 0x7f5be0070f20 2023-07-06T02:23:33.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.737+0000 7f5be7cc0700 1 -- 172.21.15.89:0/3616087349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 msgr2=0x7f5be00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:33.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.737+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/3616087349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00713c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009840 tx=0x7f5bdc009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:33.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.737+0000 7f5be7cc0700 1 -- 172.21.15.89:0/3616087349 shutdown_connections 2023-07-06T02:23:33.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.737+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/3616087349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00713c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.737+0000 7f5be7cc0700 1 -- 172.21.15.89:0/3616087349 >> 172.21.15.89:0/3616087349 conn(0x7f5be00cf1c0 msgr2=0x7f5be00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.737+0000 7f5be7cc0700 1 -- 172.21.15.89:0/3616087349 shutdown_connections 2023-07-06T02:23:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.737+0000 7f5be7cc0700 1 -- 172.21.15.89:0/3616087349 wait complete. 2023-07-06T02:23:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.739+0000 7f5be7cc0700 1 Processor -- start 2023-07-06T02:23:33.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.739+0000 7f5be7cc0700 1 -- start start 2023-07-06T02:23:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.739+0000 7f5be7cc0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be7cc0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be015c500 0x7f5be00b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be7cc0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5be0160000 0x7f5be015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be7cc0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00aee80 con 0x7f5be0070f20 2023-07-06T02:23:33.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be7cc0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be014ffa0 con 0x7f5be015c500 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be7cc0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00aeb30 con 0x7f5be0160000 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33326/0 (socket says 172.21.15.89:33326) 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 -- 172.21.15.89:0/706637721 learned_addr learned my addr 172.21.15.89:0/706637721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be525b700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be015c500 0x7f5be00b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be625d700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5be0160000 0x7f5be015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 -- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5be0160000 msgr2=0x7f5be015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:33.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5be0160000 0x7f5be015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 -- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be015c500 msgr2=0x7f5be00b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be015c500 0x7f5be00b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 -- 172.21.15.89:0/706637721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0094a0 con 0x7f5be0070f20 2023-07-06T02:23:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be5a5c700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00b3ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c4e0 tx=0x7f5bd000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5bd67fc700 1 -- 172.21.15.89:0/706637721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00052d0 con 0x7f5be0070f20 2023-07-06T02:23:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5bd67fc700 1 -- 172.21.15.89:0/706637721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0005430 con 0x7f5be0070f20 2023-07-06T02:23:33.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be00b2910 con 0x7f5be0070f20 2023-07-06T02:23:33.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5bd67fc700 1 -- 172.21.15.89:0/706637721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00038c0 con 0x7f5be0070f20 2023-07-06T02:23:33.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.740+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be00b2ed0 con 0x7f5be0070f20 2023-07-06T02:23:33.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.741+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0074270 con 0x7f5be0070f20 2023-07-06T02:23:33.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.742+0000 7f5bd67fc700 1 -- 172.21.15.89:0/706637721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5bd0003a20 con 0x7f5be0070f20 2023-07-06T02:23:33.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.742+0000 7f5bd67fc700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5bc406ace0 0x7f5bc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:33.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.742+0000 7f5bd67fc700 1 -- 172.21.15.89:0/706637721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5bd0088c50 con 0x7f5be0070f20 2023-07-06T02:23:33.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.742+0000 7f5be525b700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5bc406ace0 0x7f5bc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:33.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.742+0000 7f5be525b700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5bc406ace0 0x7f5bc406d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0090b0 tx=0x7f5bdc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:33.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.746+0000 7f5bd67fc700 1 -- 172.21.15.89:0/706637721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5bd008e020 con 0x7f5be0070f20 2023-07-06T02:23:33.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.936+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5be0071dd0 con 0x7f5be0070f20 2023-07-06T02:23:33.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.937+0000 7f5bd67fc700 1 -- 172.21.15.89:0/706637721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f5bd0053990 con 0x7f5be0070f20 2023-07-06T02:23:33.941 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:33.941 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:33.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5bc406ace0 msgr2=0x7f5bc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:33.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5bc406ace0 0x7f5bc406d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0090b0 tx=0x7f5bdc01c040 comp rx=0 tx=0).stop 2023-07-06T02:23:33.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 msgr2=0x7f5be00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:33.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00b3ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c4e0 tx=0x7f5bd000c7f0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 shutdown_connections 2023-07-06T02:23:33.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5be0160000 0x7f5be015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5bc406ace0 0x7f5bc406d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5be0070f20 0x7f5be00b3ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 --2- 172.21.15.89:0/706637721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be015c500 0x7f5be00b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:33.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 >> 172.21.15.89:0/706637721 conn(0x7f5be00cf1c0 msgr2=0x7f5be0160420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:33.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 shutdown_connections 2023-07-06T02:23:33.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:33.939+0000 7f5be7cc0700 1 -- 172.21.15.89:0/706637721 wait complete. 2023-07-06T02:23:33.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:33.956 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.077+0000 7f5d620ae700 1 Processor -- start 2023-07-06T02:23:34.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.077+0000 7f5d620ae700 1 -- start start 2023-07-06T02:23:34.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d620ae700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5d5c156ab0 0x7f5d5c160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d620ae700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d5c160bb0 0x7f5d5c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d620ae700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c0cb3d0 con 0x7f5d5c1600b0 2023-07-06T02:23:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d620ae700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c156e80 con 0x7f5d5c160bb0 2023-07-06T02:23:34.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d620ae700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c0ca370 con 0x7f5d5c156ab0 2023-07-06T02:23:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d5c160bb0 0x7f5d5c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d5c160bb0 0x7f5d5c0b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33334/0 (socket says 172.21.15.89:33334) 2023-07-06T02:23:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5d5c1600b0 0x7f5d5c06d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58236/0 (socket says 172.21.15.89:58236) 2023-07-06T02:23:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 -- 172.21.15.89:0/2599089729 learned_addr learned my addr 172.21.15.89:0/2599089729 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5b7fe700 1 --2- 172.21.15.89:0/2599089729 >> v2:172.21.15.172:3300/0 conn(0x7f5d5c156ab0 0x7f5d5c160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 -- 172.21.15.89:0/2599089729 >> v2:172.21.15.172:3300/0 conn(0x7f5d5c156ab0 msgr2=0x7f5d5c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 --2- 172.21.15.89:0/2599089729 >> v2:172.21.15.172:3300/0 conn(0x7f5d5c156ab0 0x7f5d5c160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5a7fc700 1 -- 172.21.15.89:0/2599089729 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498752513 0 0) 0x7f5d5c0cb3d0 con 0x7f5d5c1600b0 2023-07-06T02:23:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 -- 172.21.15.89:0/2599089729 >> v1:172.21.15.89:6789/0 conn(0x7f5d5c1600b0 legacy=0x7f5d5c06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 -- 172.21.15.89:0/2599089729 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0ca610 con 0x7f5d5c160bb0 2023-07-06T02:23:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.078+0000 7f5d5affd700 1 --2- 172.21.15.89:0/2599089729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c160bb0 0x7f5d5c0b1490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06e9e0 tx=0x7f5d4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ebd63ad2cdda21 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d5a7fc700 1 -- 172.21.15.89:0/2599089729 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c011830 con 0x7f5d5c160bb0 2023-07-06T02:23:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d5a7fc700 1 -- 172.21.15.89:0/2599089729 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c011990 con 0x7f5d5c160bb0 2023-07-06T02:23:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d5a7fc700 1 -- 172.21.15.89:0/2599089729 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c011d20 con 0x7f5d5c160bb0 2023-07-06T02:23:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 -- 172.21.15.89:0/2599089729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c160bb0 msgr2=0x7f5d5c0b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 --2- 172.21.15.89:0/2599089729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c160bb0 0x7f5d5c0b1490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06e9e0 tx=0x7f5d4c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 -- 172.21.15.89:0/2599089729 shutdown_connections 2023-07-06T02:23:34.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 --2- 172.21.15.89:0/2599089729 >> v2:172.21.15.172:3300/0 conn(0x7f5d5c156ab0 0x7f5d5c160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 --2- 172.21.15.89:0/2599089729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c160bb0 0x7f5d5c0b1490 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 -- 172.21.15.89:0/2599089729 >> 172.21.15.89:0/2599089729 conn(0x7f5d5c0cc490 msgr2=0x7f5d5c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:34.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 -- 172.21.15.89:0/2599089729 shutdown_connections 2023-07-06T02:23:34.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.079+0000 7f5d620ae700 1 -- 172.21.15.89:0/2599089729 wait complete. 2023-07-06T02:23:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.080+0000 7f5d620ae700 1 Processor -- start 2023-07-06T02:23:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.080+0000 7f5d620ae700 1 -- start start 2023-07-06T02:23:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d620ae700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d5c156ab0 0x7f5d5c155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d620ae700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c1600b0 0x7f5d5c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d620ae700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d5c0acce0 0x7f5d5c06c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d620ae700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0ac8c0 con 0x7f5d5c1600b0 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d620ae700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0abc40 con 0x7f5d5c0acce0 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d620ae700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0abf20 con 0x7f5d5c156ab0 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c1600b0 0x7f5d5c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c1600b0 0x7f5d5c06bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33340/0 (socket says 172.21.15.89:33340) 2023-07-06T02:23:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.081+0000 7f5d5affd700 1 -- 172.21.15.89:0/1157049962 learned_addr learned my addr 172.21.15.89:0/1157049962 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5bfff700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d5c0acce0 0x7f5d5c06c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5b7fe700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d5c156ab0 0x7f5d5c155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5affd700 1 -- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d5c156ab0 msgr2=0x7f5d5c155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5affd700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d5c156ab0 0x7f5d5c155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5affd700 1 -- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d5c0acce0 msgr2=0x7f5d5c06c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5affd700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d5c0acce0 0x7f5d5c06c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5affd700 1 -- 172.21.15.89:0/1157049962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009550 con 0x7f5d5c1600b0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d5affd700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c1600b0 0x7f5d5c06bb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009c10 tx=0x7f5d4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d43fff700 1 -- 172.21.15.89:0/1157049962 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0036a0 con 0x7f5d5c1600b0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d43fff700 1 -- 172.21.15.89:0/1157049962 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c003800 con 0x7f5d5c1600b0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c0ac200 con 0x7f5d5c1600b0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d43fff700 1 -- 172.21.15.89:0/1157049962 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c003b70 con 0x7f5d5c1600b0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.082+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c0ab8f0 con 0x7f5d5c1600b0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.083+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c0cbe20 con 0x7f5d5c1600b0 2023-07-06T02:23:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.083+0000 7f5d43fff700 1 -- 172.21.15.89:0/1157049962 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d4c003cd0 con 0x7f5d5c1600b0 2023-07-06T02:23:34.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.084+0000 7f5d43fff700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d4806ace0 0x7f5d4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.084+0000 7f5d43fff700 1 -- 172.21.15.89:0/1157049962 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5d4c021030 con 0x7f5d5c1600b0 2023-07-06T02:23:34.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.084+0000 7f5d5b7fe700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d4806ace0 0x7f5d4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.084+0000 7f5d5b7fe700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d4806ace0 0x7f5d4806d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0b5960 tx=0x7f5d50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.088+0000 7f5d43fff700 1 -- 172.21.15.89:0/1157049962 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d4c056070 con 0x7f5d5c1600b0 2023-07-06T02:23:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.275+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d5c066fb0 con 0x7f5d5c1600b0 2023-07-06T02:23:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.275+0000 7f5d43fff700 1 -- 172.21.15.89:0/1157049962 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d4c059690 con 0x7f5d5c1600b0 2023-07-06T02:23:34.278 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:34.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d4806ace0 msgr2=0x7f5d4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d4806ace0 0x7f5d4806d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0b5960 tx=0x7f5d50008040 comp rx=0 tx=0).stop 2023-07-06T02:23:34.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c1600b0 msgr2=0x7f5d5c06bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c1600b0 0x7f5d5c06bb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009c10 tx=0x7f5d4c004750 comp rx=0 tx=0).stop 2023-07-06T02:23:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 shutdown_connections 2023-07-06T02:23:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d5c0acce0 0x7f5d5c06c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d5c156ab0 0x7f5d5c155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d4806ace0 0x7f5d4806d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 --2- 172.21.15.89:0/1157049962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d5c1600b0 0x7f5d5c06bb00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 >> 172.21.15.89:0/1157049962 conn(0x7f5d5c0cc490 msgr2=0x7f5d5c0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 shutdown_connections 2023-07-06T02:23:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.277+0000 7f5d620ae700 1 -- 172.21.15.89:0/1157049962 wait complete. 2023-07-06T02:23:34.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:34.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.413+0000 7fca66285700 1 Processor -- start 2023-07-06T02:23:34.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.413+0000 7fca66285700 1 -- start start 2023-07-06T02:23:34.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.414+0000 7fca66285700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fca60160710 0x7fca600713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.414+0000 7fca66285700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca600cb9f0 con 0x7fca60070f20 2023-07-06T02:23:34.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.414+0000 7fca66285700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6015d8e0 con 0x7fca6015c500 2023-07-06T02:23:34.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.414+0000 7fca66285700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca60071900 con 0x7fca60160710 2023-07-06T02:23:34.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.414+0000 7fca5f7fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fca60070f20 0x7fca60160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58248/0 (socket says 172.21.15.89:58248) 2023-07-06T02:23:34.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.414+0000 7fca5f7fe700 1 -- 172.21.15.89:0/4055003071 learned_addr learned my addr 172.21.15.89:0/4055003071 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:34.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca64822700 1 --2- 172.21.15.89:0/4055003071 >> v2:172.21.15.172:3300/0 conn(0x7fca60160710 0x7fca600713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140476874 0 0) 0x7fca600cb9f0 con 0x7fca60070f20 2023-07-06T02:23:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca480035b0 con 0x7fca60070f20 2023-07-06T02:23:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3471567623 0 0) 0x7fca480035b0 con 0x7fca60070f20 2023-07-06T02:23:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 >> v2:172.21.15.172:3300/0 conn(0x7fca60160710 msgr2=0x7fca600713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 --2- 172.21.15.89:0/4055003071 >> v2:172.21.15.172:3300/0 conn(0x7fca60160710 0x7fca600713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 >> v1:172.21.15.172:6789/0 conn(0x7fca6015c500 legacy=0x7fca6006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60160be0 con 0x7fca60070f20 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fca50001670 con 0x7fca60070f20 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 >> v1:172.21.15.89:6789/0 conn(0x7fca60070f20 legacy=0x7fca60160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca48003a00 0x7fca48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca480043f0 0x7fca480048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca480050b0 0x7fca4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4800c000 con 0x7fca480043f0 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4800c140 con 0x7fca48003a00 2023-07-06T02:23:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4800c300 con 0x7fca480050b0 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca64822700 1 --2- 172.21.15.89:0/4055003071 >> v2:172.21.15.172:3300/0 conn(0x7fca60160710 0x7fca600713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5f7fe700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca480043f0 0x7fca480048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5ffff700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca48003a00 0x7fca48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca64822700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca480050b0 0x7fca4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5f7fe700 1 -- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca480050b0 msgr2=0x7fca4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5f7fe700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca480050b0 0x7fca4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5f7fe700 1 -- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca48003a00 msgr2=0x7fca48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5f7fe700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca48003a00 0x7fca48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca5f7fe700 1 -- 172.21.15.89:0/4055003071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480035b0 con 0x7fca480043f0 2023-07-06T02:23:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.415+0000 7fca64822700 1 -- 172.21.15.89:0/4055003071 reap_dead start 2023-07-06T02:23:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca5f7fe700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca480043f0 0x7fca480048b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c2f0 tx=0x7fca48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede5c6d8d9304be3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000ccf0 con 0x7fca480043f0 2023-07-06T02:23:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5000ce70 con 0x7fca480043f0 2023-07-06T02:23:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca5effd700 1 -- 172.21.15.89:0/4055003071 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50005b80 con 0x7fca480043f0 2023-07-06T02:23:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca66285700 1 -- 172.21.15.89:0/4055003071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca480043f0 msgr2=0x7fca480048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca66285700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca480043f0 0x7fca480048b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c2f0 tx=0x7fca48005880 comp rx=0 tx=0).stop 2023-07-06T02:23:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca66285700 1 -- 172.21.15.89:0/4055003071 shutdown_connections 2023-07-06T02:23:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca66285700 1 --2- 172.21.15.89:0/4055003071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca480043f0 0x7fca480048b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca66285700 1 -- 172.21.15.89:0/4055003071 >> 172.21.15.89:0/4055003071 conn(0x7fca600cf1c0 msgr2=0x7fca600cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca66285700 1 -- 172.21.15.89:0/4055003071 shutdown_connections 2023-07-06T02:23:34.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.416+0000 7fca66285700 1 -- 172.21.15.89:0/4055003071 wait complete. 2023-07-06T02:23:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.417+0000 7fca66285700 1 Processor -- start 2023-07-06T02:23:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca66285700 1 -- start start 2023-07-06T02:23:34.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca66285700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca480043f0 0x7fca600b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca66285700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca600b3a70 0x7fca6015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca66285700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca600b0c00 0x7fca600b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca66285700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca600c3df0 con 0x7fca600b3a70 2023-07-06T02:23:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca66285700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca600b15c0 con 0x7fca480043f0 2023-07-06T02:23:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca66285700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6015a900 con 0x7fca600b0c00 2023-07-06T02:23:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca5f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca600b3a70 0x7fca6015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca5f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca600b3a70 0x7fca6015a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33360/0 (socket says 172.21.15.89:33360) 2023-07-06T02:23:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.418+0000 7fca5f7fe700 1 -- 172.21.15.89:0/807004939 learned_addr learned my addr 172.21.15.89:0/807004939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5ffff700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca480043f0 0x7fca600b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5f7fe700 1 -- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca600b0c00 msgr2=0x7fca600b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca64822700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca600b0c00 0x7fca600b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5f7fe700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca600b0c00 0x7fca600b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5f7fe700 1 -- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca480043f0 msgr2=0x7fca600b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5f7fe700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca480043f0 0x7fca600b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5f7fe700 1 -- 172.21.15.89:0/807004939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca600b4210 con 0x7fca600b3a70 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5ffff700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca480043f0 0x7fca600b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca64822700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca600b0c00 0x7fca600b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5f7fe700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca600b3a70 0x7fca6015a3c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c8e0 tx=0x7fca50001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5cff9700 1 -- 172.21.15.89:0/807004939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50001640 con 0x7fca600b3a70 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5cff9700 1 -- 172.21.15.89:0/807004939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca500070d0 con 0x7fca600b3a70 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca600b44f0 con 0x7fca600b3a70 2023-07-06T02:23:34.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca5cff9700 1 -- 172.21.15.89:0/807004939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500074a0 con 0x7fca600b3a70 2023-07-06T02:23:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.419+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca600b4ab0 con 0x7fca600b3a70 2023-07-06T02:23:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.420+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca600c3b50 con 0x7fca600b3a70 2023-07-06T02:23:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.421+0000 7fca5cff9700 1 -- 172.21.15.89:0/807004939 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca50007600 con 0x7fca600b3a70 2023-07-06T02:23:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.421+0000 7fca5cff9700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fca4c06ace0 0x7fca4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.421+0000 7fca5cff9700 1 -- 172.21.15.89:0/807004939 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fca5008a510 con 0x7fca600b3a70 2023-07-06T02:23:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.421+0000 7fca5ffff700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fca4c06ace0 0x7fca4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.422+0000 7fca5ffff700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fca4c06ace0 0x7fca4c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca540010e0 tx=0x7fca54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.425+0000 7fca5cff9700 1 -- 172.21.15.89:0/807004939 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fca50055360 con 0x7fca600b3a70 2023-07-06T02:23:34.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.612+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca600c3b50 con 0x7fca600b3a70 2023-07-06T02:23:34.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.612+0000 7fca5cff9700 1 -- 172.21.15.89:0/807004939 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca600c3b50 con 0x7fca600b3a70 2023-07-06T02:23:34.615 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fca4c06ace0 msgr2=0x7fca4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fca4c06ace0 0x7fca4c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca540010e0 tx=0x7fca54009040 comp rx=0 tx=0).stop 2023-07-06T02:23:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca600b3a70 msgr2=0x7fca6015a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca600b3a70 0x7fca6015a3c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c8e0 tx=0x7fca50001a00 comp rx=0 tx=0).stop 2023-07-06T02:23:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 shutdown_connections 2023-07-06T02:23:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca600b0c00 0x7fca600b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca480043f0 0x7fca600b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fca4c06ace0 0x7fca4c06d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 --2- 172.21.15.89:0/807004939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca600b3a70 0x7fca6015a3c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 >> 172.21.15.89:0/807004939 conn(0x7fca600cf1c0 msgr2=0x7fca6015db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 shutdown_connections 2023-07-06T02:23:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.614+0000 7fca66285700 1 -- 172.21.15.89:0/807004939 wait complete. 2023-07-06T02:23:34.629 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:34.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.749+0000 7fea4051d700 1 Processor -- start 2023-07-06T02:23:34.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.749+0000 7fea4051d700 1 -- start start 2023-07-06T02:23:34.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea4051d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea381600b0 0x7fea380b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea4051d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea4051d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380cb3d0 con 0x7fea38156ab0 2023-07-06T02:23:34.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea4051d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea380b1e10 con 0x7fea381600b0 2023-07-06T02:23:34.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea4051d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea38156e80 con 0x7fea380b19d0 2023-07-06T02:23:34.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea3dab8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea381600b0 0x7fea380b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea3eaba700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea3dab8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea381600b0 0x7fea380b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:60356/0 (socket says 172.21.15.89:60356) 2023-07-06T02:23:34.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea3dab8700 1 -- 172.21.15.89:0/3764181434 learned_addr learned my addr 172.21.15.89:0/3764181434 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:34.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.750+0000 7fea3dab8700 1 -- 172.21.15.89:0/3764181434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 msgr2=0x7fea380cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:34.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea3dab8700 1 --2- 172.21.15.89:0/3764181434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea3dab8700 1 -- 172.21.15.89:0/3764181434 >> v1:172.21.15.172:6789/0 conn(0x7fea38156ab0 legacy=0x7fea38160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea3dab8700 1 -- 172.21.15.89:0/3764181434 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380ca370 con 0x7fea381600b0 2023-07-06T02:23:34.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea3dab8700 1 --2- 172.21.15.89:0/3764181434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea381600b0 0x7fea380b1490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea280098f0 tx=0x7fea28009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6577c2cb9ebd5b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea3d2b7700 1 -- 172.21.15.89:0/3764181434 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011830 con 0x7fea381600b0 2023-07-06T02:23:34.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea3d2b7700 1 -- 172.21.15.89:0/3764181434 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28011990 con 0x7fea381600b0 2023-07-06T02:23:34.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea3d2b7700 1 -- 172.21.15.89:0/3764181434 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011d20 con 0x7fea381600b0 2023-07-06T02:23:34.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea4051d700 1 -- 172.21.15.89:0/3764181434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea381600b0 msgr2=0x7fea380b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.751+0000 7fea4051d700 1 --2- 172.21.15.89:0/3764181434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea381600b0 0x7fea380b1490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea280098f0 tx=0x7fea28009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.752+0000 7fea4051d700 1 -- 172.21.15.89:0/3764181434 shutdown_connections 2023-07-06T02:23:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.752+0000 7fea4051d700 1 --2- 172.21.15.89:0/3764181434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.752+0000 7fea4051d700 1 --2- 172.21.15.89:0/3764181434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea381600b0 0x7fea380b1490 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.752+0000 7fea4051d700 1 -- 172.21.15.89:0/3764181434 >> 172.21.15.89:0/3764181434 conn(0x7fea380cc490 msgr2=0x7fea380cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.752+0000 7fea4051d700 1 -- 172.21.15.89:0/3764181434 shutdown_connections 2023-07-06T02:23:34.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.752+0000 7fea4051d700 1 -- 172.21.15.89:0/3764181434 wait complete. 2023-07-06T02:23:34.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 Processor -- start 2023-07-06T02:23:34.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 -- start start 2023-07-06T02:23:34.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea38156ab0 0x7fea3814f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea381600b0 0x7fea3814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea380b3740 con 0x7fea381600b0 2023-07-06T02:23:34.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea380b2bc0 con 0x7fea38156ab0 2023-07-06T02:23:34.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.753+0000 7fea4051d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea380b2ea0 con 0x7fea380b19d0 2023-07-06T02:23:34.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea381600b0 0x7fea3814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea381600b0 0x7fea3814fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33374/0 (socket says 172.21.15.89:33374) 2023-07-06T02:23:34.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 -- 172.21.15.89:0/366936959 learned_addr learned my addr 172.21.15.89:0/366936959 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:34.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3dab8700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea38156ab0 0x7fea3814f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3e2b9700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 -- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 msgr2=0x7fea380c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 -- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea38156ab0 msgr2=0x7fea3814f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea38156ab0 0x7fea3814f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 -- 172.21.15.89:0/366936959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea28009550 con 0x7fea381600b0 2023-07-06T02:23:34.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea3eaba700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea381600b0 0x7fea3814fb00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fea3000efc0 tx=0x7fea3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea2effd700 1 -- 172.21.15.89:0/366936959 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30009ea0 con 0x7fea381600b0 2023-07-06T02:23:34.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea2effd700 1 -- 172.21.15.89:0/366936959 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3000f040 con 0x7fea381600b0 2023-07-06T02:23:34.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea380b3180 con 0x7fea381600b0 2023-07-06T02:23:34.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea2effd700 1 -- 172.21.15.89:0/366936959 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000ced0 con 0x7fea381600b0 2023-07-06T02:23:34.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.754+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea380c9850 con 0x7fea381600b0 2023-07-06T02:23:34.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.755+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea380c9a40 con 0x7fea381600b0 2023-07-06T02:23:34.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.756+0000 7fea2effd700 1 -- 172.21.15.89:0/366936959 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea30014480 con 0x7fea381600b0 2023-07-06T02:23:34.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.756+0000 7fea2effd700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fea1c06ace0 0x7fea1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:34.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.756+0000 7fea2effd700 1 -- 172.21.15.89:0/366936959 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fea30089f70 con 0x7fea381600b0 2023-07-06T02:23:34.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.756+0000 7fea3e2b9700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fea1c06ace0 0x7fea1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:34.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.757+0000 7fea3e2b9700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fea1c06ace0 0x7fea1c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea34007fd0 tx=0x7fea34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:34.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.761+0000 7fea2effd700 1 -- 172.21.15.89:0/366936959 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fea300bc310 con 0x7fea381600b0 2023-07-06T02:23:34.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.945+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea38074270 con 0x7fea381600b0 2023-07-06T02:23:34.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.946+0000 7fea2effd700 1 -- 172.21.15.89:0/366936959 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea30018020 con 0x7fea381600b0 2023-07-06T02:23:34.949 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:34.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fea1c06ace0 msgr2=0x7fea1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fea1c06ace0 0x7fea1c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea34007fd0 tx=0x7fea34009040 comp rx=0 tx=0).stop 2023-07-06T02:23:34.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea381600b0 msgr2=0x7fea3814fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:34.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea381600b0 0x7fea3814fb00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fea3000efc0 tx=0x7fea3000c3a0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 shutdown_connections 2023-07-06T02:23:34.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fea1c06ace0 0x7fea1c06d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea381600b0 0x7fea3814fb00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea380b19d0 0x7fea380c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 --2- 172.21.15.89:0/366936959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea38156ab0 0x7fea3814f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:34.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 >> 172.21.15.89:0/366936959 conn(0x7fea380cc490 msgr2=0x7fea380cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:34.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 shutdown_connections 2023-07-06T02:23:34.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:34.948+0000 7fea4051d700 1 -- 172.21.15.89:0/366936959 wait complete. 2023-07-06T02:23:34.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:35.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.083+0000 7f9a5c435700 1 Processor -- start 2023-07-06T02:23:35.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.083+0000 7f9a5c435700 1 -- start start 2023-07-06T02:23:35.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a5c435700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a5406dc10 0x7f9a5415a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a5c435700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a540cd350 con 0x7f9a5415adb0 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a5c435700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5406e110 con 0x7f9a540705a0 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a5c435700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5406e3f0 con 0x7f9a5406dc10 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a5a9d2700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9a5415adb0 0x7f9a54157950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58284/0 (socket says 172.21.15.89:58284) 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a5a9d2700 1 -- 172.21.15.89:0/3391268103 learned_addr learned my addr 172.21.15.89:0/3391268103 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224072318 0 0) 0x7f9a540cd350 con 0x7f9a5415adb0 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a599d0700 1 --2- 172.21.15.89:0/3391268103 >> v2:172.21.15.172:3300/0 conn(0x7f9a5406dc10 0x7f9a5415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.084+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a400035b0 con 0x7f9a5415adb0 2023-07-06T02:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 748807442 0 0) 0x7f9a400035b0 con 0x7f9a5415adb0 2023-07-06T02:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 >> v2:172.21.15.172:3300/0 conn(0x7f9a5406dc10 msgr2=0x7f9a5415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 --2- 172.21.15.89:0/3391268103 >> v2:172.21.15.172:3300/0 conn(0x7f9a5406dc10 0x7f9a5415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 >> v1:172.21.15.172:6789/0 conn(0x7f9a540705a0 legacy=0x7f9a540ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5406e6d0 con 0x7f9a5415adb0 2023-07-06T02:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f9a4c0019f0 con 0x7f9a5415adb0 2023-07-06T02:23:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 >> v1:172.21.15.89:6789/0 conn(0x7f9a5415adb0 legacy=0x7f9a54157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a40003a30 0x7f9a40003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40004850 0x7f9a40004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a40005200 0x7f9a4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5406e3f0 con 0x7f9a40005200 2023-07-06T02:23:35.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4000c0e0 con 0x7f9a40004850 2023-07-06T02:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4000c270 con 0x7f9a40003a30 2023-07-06T02:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a40005200 0x7f9a4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a599d0700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40004850 0x7f9a40004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a1d1700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a40003a30 0x7f9a40003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 -- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a40003a30 msgr2=0x7f9a40003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a40003a30 0x7f9a40003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 -- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40004850 msgr2=0x7f9a40004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40004850 0x7f9a40004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 -- 172.21.15.89:0/3391268103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a400035b0 con 0x7f9a40005200 2023-07-06T02:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a1d1700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a40003a30 0x7f9a40003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 -- 172.21.15.89:0/3391268103 reap_dead start 2023-07-06T02:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a5a9d2700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a40005200 0x7f9a4000ba10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00a410 tx=0x7f9a40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163f2110bc2eefe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c011880 con 0x7f9a40005200 2023-07-06T02:23:35.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0119e0 con 0x7f9a40005200 2023-07-06T02:23:35.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.085+0000 7f9a591cf700 1 -- 172.21.15.89:0/3391268103 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c011d90 con 0x7f9a40005200 2023-07-06T02:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.086+0000 7f9a5c435700 1 -- 172.21.15.89:0/3391268103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a40005200 msgr2=0x7f9a4000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.086+0000 7f9a5c435700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a40005200 0x7f9a4000ba10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00a410 tx=0x7f9a40005880 comp rx=0 tx=0).stop 2023-07-06T02:23:35.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.086+0000 7f9a5c435700 1 -- 172.21.15.89:0/3391268103 shutdown_connections 2023-07-06T02:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.086+0000 7f9a5c435700 1 --2- 172.21.15.89:0/3391268103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a40005200 0x7f9a4000ba10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.086+0000 7f9a5c435700 1 -- 172.21.15.89:0/3391268103 >> 172.21.15.89:0/3391268103 conn(0x7f9a540d0b00 msgr2=0x7f9a540d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.086+0000 7f9a5c435700 1 -- 172.21.15.89:0/3391268103 shutdown_connections 2023-07-06T02:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.086+0000 7f9a5c435700 1 -- 172.21.15.89:0/3391268103 wait complete. 2023-07-06T02:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.087+0000 7f9a5c435700 1 Processor -- start 2023-07-06T02:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.087+0000 7f9a5c435700 1 -- start start 2023-07-06T02:23:35.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5c435700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40005200 0x7f9a540c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5c435700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5415d4a0 0x7f9a541475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5c435700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a54157f60 0x7f9a541583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5c435700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5414b4b0 con 0x7f9a5415d4a0 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5c435700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54158910 con 0x7f9a40005200 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5c435700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54145980 con 0x7f9a54157f60 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a599d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5415d4a0 0x7f9a541475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a599d0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5415d4a0 0x7f9a541475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33392/0 (socket says 172.21.15.89:33392) 2023-07-06T02:23:35.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a599d0700 1 -- 172.21.15.89:0/2755166322 learned_addr learned my addr 172.21.15.89:0/2755166322 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5a1d1700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40005200 0x7f9a540c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.088+0000 7f9a5a9d2700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a54157f60 0x7f9a541583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a599d0700 1 -- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a54157f60 msgr2=0x7f9a541583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a599d0700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a54157f60 0x7f9a541583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a599d0700 1 -- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40005200 msgr2=0x7f9a540c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a599d0700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40005200 0x7f9a540c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a599d0700 1 -- 172.21.15.89:0/2755166322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54147f00 con 0x7f9a5415d4a0 2023-07-06T02:23:35.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a599d0700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5415d4a0 0x7f9a541475b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9a500119a0 tx=0x7f9a50011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a4affd700 1 -- 172.21.15.89:0/2755166322 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000f940 con 0x7f9a5415d4a0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a4affd700 1 -- 172.21.15.89:0/2755166322 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5000faa0 con 0x7f9a5415d4a0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a4affd700 1 -- 172.21.15.89:0/2755166322 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000fe30 con 0x7f9a5415d4a0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a541481e0 con 0x7f9a5415d4a0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.089+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a54145610 con 0x7f9a5415d4a0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.090+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a54074270 con 0x7f9a5415d4a0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.090+0000 7f9a4affd700 1 -- 172.21.15.89:0/2755166322 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a50014440 con 0x7f9a5415d4a0 2023-07-06T02:23:35.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.091+0000 7f9a4affd700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9a3806ace0 0x7f9a3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.091+0000 7f9a4affd700 1 -- 172.21.15.89:0/2755166322 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9a50089db0 con 0x7f9a5415d4a0 2023-07-06T02:23:35.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.091+0000 7f9a5a1d1700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9a3806ace0 0x7f9a3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.091+0000 7f9a5a1d1700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9a3806ace0 0x7f9a3806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a44000c00 tx=0x7f9a44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.096+0000 7f9a4affd700 1 -- 172.21.15.89:0/2755166322 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a50054c20 con 0x7f9a5415d4a0 2023-07-06T02:23:35.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.281+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a5406f590 con 0x7f9a5415d4a0 2023-07-06T02:23:35.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.281+0000 7f9a4affd700 1 -- 172.21.15.89:0/2755166322 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a50058240 con 0x7f9a5415d4a0 2023-07-06T02:23:35.328 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:35.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.283+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9a3806ace0 msgr2=0x7f9a3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.283+0000 7f9a5c435700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9a3806ace0 0x7f9a3806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a44000c00 tx=0x7f9a44009040 comp rx=0 tx=0).stop 2023-07-06T02:23:35.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.283+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5415d4a0 msgr2=0x7f9a541475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.283+0000 7f9a5c435700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5415d4a0 0x7f9a541475b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9a500119a0 tx=0x7f9a50011cb0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.283+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 shutdown_connections 2023-07-06T02:23:35.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.283+0000 7f9a5c435700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a54157f60 0x7f9a541583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.284+0000 7f9a5c435700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a40005200 0x7f9a540c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.284+0000 7f9a5c435700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9a3806ace0 0x7f9a3806d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.284+0000 7f9a5c435700 1 --2- 172.21.15.89:0/2755166322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a5415d4a0 0x7f9a541475b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.284+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 >> 172.21.15.89:0/2755166322 conn(0x7f9a540d0b00 msgr2=0x7f9a54159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:35.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.284+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 shutdown_connections 2023-07-06T02:23:35.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.284+0000 7f9a5c435700 1 -- 172.21.15.89:0/2755166322 wait complete. 2023-07-06T02:23:35.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:35.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.452+0000 7fe791a65700 1 Processor -- start 2023-07-06T02:23:35.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe791a65700 1 -- start start 2023-07-06T02:23:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe791a65700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe78c0b1490 0x7fe78c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe791a65700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe78c0cc000 0x7fe78c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe791a65700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c15de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe791a65700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c0b1da0 con 0x7fe78c0b1490 2023-07-06T02:23:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe791a65700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c06a990 con 0x7fe78c0cc000 2023-07-06T02:23:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe791a65700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c06ac70 con 0x7fe78c06b220 2023-07-06T02:23:35.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe78b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe78c0b1490 0x7fe78c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe78b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe78c0b1490 0x7fe78c0caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33400/0 (socket says 172.21.15.89:33400) 2023-07-06T02:23:35.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.453+0000 7fe78b7fe700 1 -- 172.21.15.89:0/98510750 learned_addr learned my addr 172.21.15.89:0/98510750 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78affd700 1 --2- 172.21.15.89:0/98510750 >> v2:172.21.15.172:3300/0 conn(0x7fe78c0cc000 0x7fe78c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78bfff700 1 --2- 172.21.15.89:0/98510750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78b7fe700 1 -- 172.21.15.89:0/98510750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 msgr2=0x7fe78c15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/98510750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78b7fe700 1 -- 172.21.15.89:0/98510750 >> v2:172.21.15.172:3300/0 conn(0x7fe78c0cc000 msgr2=0x7fe78c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/98510750 >> v2:172.21.15.172:3300/0 conn(0x7fe78c0cc000 0x7fe78c0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78b7fe700 1 -- 172.21.15.89:0/98510750 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c06d170 con 0x7fe78c0b1490 2023-07-06T02:23:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/98510750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0b1490 0x7fe78c0caeb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0b2600 tx=0x7fe77c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157ba20879a787a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78a7fc700 1 -- 172.21.15.89:0/98510750 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00b870 con 0x7fe78c0b1490 2023-07-06T02:23:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78a7fc700 1 -- 172.21.15.89:0/98510750 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c00b9d0 con 0x7fe78c0b1490 2023-07-06T02:23:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.454+0000 7fe78a7fc700 1 -- 172.21.15.89:0/98510750 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00bd60 con 0x7fe78c0b1490 2023-07-06T02:23:35.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 -- 172.21.15.89:0/98510750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0b1490 msgr2=0x7fe78c0caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 --2- 172.21.15.89:0/98510750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0b1490 0x7fe78c0caeb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0b2600 tx=0x7fe77c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 -- 172.21.15.89:0/98510750 shutdown_connections 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 --2- 172.21.15.89:0/98510750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 --2- 172.21.15.89:0/98510750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0b1490 0x7fe78c0caeb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 --2- 172.21.15.89:0/98510750 >> v2:172.21.15.172:3300/0 conn(0x7fe78c0cc000 0x7fe78c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 -- 172.21.15.89:0/98510750 >> 172.21.15.89:0/98510750 conn(0x7fe78c0cc490 msgr2=0x7fe78c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 -- 172.21.15.89:0/98510750 shutdown_connections 2023-07-06T02:23:35.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.455+0000 7fe791a65700 1 -- 172.21.15.89:0/98510750 wait complete. 2023-07-06T02:23:35.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.456+0000 7fe791a65700 1 Processor -- start 2023-07-06T02:23:35.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.456+0000 7fe791a65700 1 -- start start 2023-07-06T02:23:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe791a65700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe791a65700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe78c0b1490 0x7fe78c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe791a65700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0cc000 0x7fe78c14aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe791a65700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c15c620 con 0x7fe78c0cc000 2023-07-06T02:23:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe791a65700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c15c900 con 0x7fe78c0b1490 2023-07-06T02:23:35.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe791a65700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c15cbe0 con 0x7fe78c06b220 2023-07-06T02:23:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0cc000 0x7fe78c14aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0cc000 0x7fe78c14aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33408/0 (socket says 172.21.15.89:33408) 2023-07-06T02:23:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.457+0000 7fe78bfff700 1 -- 172.21.15.89:0/2873323923 learned_addr learned my addr 172.21.15.89:0/2873323923 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78affd700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe78c0b1490 0x7fe78c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78bfff700 1 -- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 msgr2=0x7fe78c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78bfff700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78bfff700 1 -- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe78c0b1490 msgr2=0x7fe78c15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78bfff700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe78c0b1490 0x7fe78c15bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78bfff700 1 -- 172.21.15.89:0/2873323923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009550 con 0x7fe78c0cc000 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe78bfff700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0cc000 0x7fe78c14aa30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe784007e40 tx=0x7fe78400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe773fff700 1 -- 172.21.15.89:0/2873323923 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400eea0 con 0x7fe78c0cc000 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe773fff700 1 -- 172.21.15.89:0/2873323923 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784014420 con 0x7fe78c0cc000 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c15cec0 con 0x7fe78c0cc000 2023-07-06T02:23:35.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe773fff700 1 -- 172.21.15.89:0/2873323923 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7840147b0 con 0x7fe78c0cc000 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.458+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c15d6c0 con 0x7fe78c0cc000 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.459+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c15d3e0 con 0x7fe78c0cc000 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.459+0000 7fe773fff700 1 -- 172.21.15.89:0/2873323923 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe784014910 con 0x7fe78c0cc000 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.460+0000 7fe773fff700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe77806ace0 0x7fe77806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.460+0000 7fe773fff700 1 -- 172.21.15.89:0/2873323923 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe78408a030 con 0x7fe78c0cc000 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.460+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe77806ace0 0x7fe77806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.460+0000 7fe78b7fe700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe77806ace0 0x7fe77806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe77c000c00 tx=0x7fe77c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.465+0000 7fe773fff700 1 -- 172.21.15.89:0/2873323923 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe7840bc310 con 0x7fe78c0cc000 2023-07-06T02:23:35.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.651+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe78c074270 con 0x7fe78c0cc000 2023-07-06T02:23:35.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.652+0000 7fe773fff700 1 -- 172.21.15.89:0/2873323923 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe78c15d3e0 con 0x7fe78c0cc000 2023-07-06T02:23:35.655 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:35.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe77806ace0 msgr2=0x7fe77806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe77806ace0 0x7fe77806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe77c000c00 tx=0x7fe77c008030 comp rx=0 tx=0).stop 2023-07-06T02:23:35.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0cc000 msgr2=0x7fe78c14aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0cc000 0x7fe78c14aa30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe784007e40 tx=0x7fe78400e3c0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 shutdown_connections 2023-07-06T02:23:35.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe77806ace0 0x7fe77806d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe78c0cc000 0x7fe78c14aa30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe78c06b220 0x7fe78c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 --2- 172.21.15.89:0/2873323923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe78c0b1490 0x7fe78c15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 >> 172.21.15.89:0/2873323923 conn(0x7fe78c0cc490 msgr2=0x7fe78c0b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:35.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 shutdown_connections 2023-07-06T02:23:35.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.654+0000 7fe791a65700 1 -- 172.21.15.89:0/2873323923 wait complete. 2023-07-06T02:23:35.669 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:35.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.787+0000 7f25647fc700 1 Processor -- start 2023-07-06T02:23:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.787+0000 7f25647fc700 1 -- start start 2023-07-06T02:23:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f25647fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f255c15c610 0x7f255c0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f25647fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c0b5e50 0x7f255c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f25647fc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f255c070870 0x7f255c0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f25647fc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c070cf0 con 0x7f255c15c610 2023-07-06T02:23:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f25647fc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0cb150 con 0x7f255c070870 2023-07-06T02:23:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f25647fc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0cb430 con 0x7f255c0b5e50 2023-07-06T02:23:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f2562598700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f255c15c610 0x7f255c0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.788+0000 7f2562598700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f255c15c610 0x7f255c0cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33412/0 (socket says 172.21.15.89:33412) 2023-07-06T02:23:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562598700 1 -- 172.21.15.89:0/2155784403 learned_addr learned my addr 172.21.15.89:0/2155784403 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562d99700 1 --2- 172.21.15.89:0/2155784403 >> v2:172.21.15.172:3300/0 conn(0x7f255c070870 0x7f255c0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2561d97700 1 --2- 172.21.15.89:0/2155784403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c0b5e50 0x7f255c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562598700 1 -- 172.21.15.89:0/2155784403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c0b5e50 msgr2=0x7f255c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562598700 1 --2- 172.21.15.89:0/2155784403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c0b5e50 0x7f255c070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562598700 1 -- 172.21.15.89:0/2155784403 >> v2:172.21.15.172:3300/0 conn(0x7f255c070870 msgr2=0x7f255c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:35.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562598700 1 --2- 172.21.15.89:0/2155784403 >> v2:172.21.15.172:3300/0 conn(0x7f255c070870 0x7f255c0ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562598700 1 -- 172.21.15.89:0/2155784403 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0b6310 con 0x7f255c15c610 2023-07-06T02:23:35.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2562598700 1 --2- 172.21.15.89:0/2155784403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c15c610 0x7f255c0cfed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f255c072720 tx=0x7f254c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a22e064a6f8af server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2561596700 1 -- 172.21.15.89:0/2155784403 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00b870 con 0x7f255c15c610 2023-07-06T02:23:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2561596700 1 -- 172.21.15.89:0/2155784403 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c00b9d0 con 0x7f255c15c610 2023-07-06T02:23:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.789+0000 7f2561596700 1 -- 172.21.15.89:0/2155784403 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00bd60 con 0x7f255c15c610 2023-07-06T02:23:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 -- 172.21.15.89:0/2155784403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c15c610 msgr2=0x7f255c0cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 --2- 172.21.15.89:0/2155784403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c15c610 0x7f255c0cfed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f255c072720 tx=0x7f254c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 -- 172.21.15.89:0/2155784403 shutdown_connections 2023-07-06T02:23:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 --2- 172.21.15.89:0/2155784403 >> v2:172.21.15.172:3300/0 conn(0x7f255c070870 0x7f255c0ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 --2- 172.21.15.89:0/2155784403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c15c610 0x7f255c0cfed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 --2- 172.21.15.89:0/2155784403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c0b5e50 0x7f255c070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 -- 172.21.15.89:0/2155784403 >> 172.21.15.89:0/2155784403 conn(0x7f255c0d0b00 msgr2=0x7f255c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 -- 172.21.15.89:0/2155784403 shutdown_connections 2023-07-06T02:23:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.790+0000 7f25647fc700 1 -- 172.21.15.89:0/2155784403 wait complete. 2023-07-06T02:23:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.791+0000 7f25647fc700 1 Processor -- start 2023-07-06T02:23:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.791+0000 7f25647fc700 1 -- start start 2023-07-06T02:23:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255c070870 0x7f255c0af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c0b5e50 0x7f255c150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c15c610 0x7f255c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c15eae0 con 0x7f255c0b5e50 2023-07-06T02:23:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c48d0 con 0x7f255c070870 2023-07-06T02:23:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c4bb0 con 0x7f255c15c610 2023-07-06T02:23:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c0b5e50 0x7f255c150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c0b5e50 0x7f255c150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33420/0 (socket says 172.21.15.89:33420) 2023-07-06T02:23:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 -- 172.21.15.89:0/4081575532 learned_addr learned my addr 172.21.15.89:0/4081575532 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2562d99700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c15c610 0x7f255c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2562598700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255c070870 0x7f255c0af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 -- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c15c610 msgr2=0x7f255c0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c15c610 0x7f255c0c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 -- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255c070870 msgr2=0x7f255c0af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255c070870 0x7f255c0af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 -- 172.21.15.89:0/4081575532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c009550 con 0x7f255c0b5e50 2023-07-06T02:23:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2561d97700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c0b5e50 0x7f255c150450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f255800e810 tx=0x7f255800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2552ffd700 1 -- 172.21.15.89:0/4081575532 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800c070 con 0x7f255c0b5e50 2023-07-06T02:23:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2552ffd700 1 -- 172.21.15.89:0/4081575532 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2558013610 con 0x7f255c0b5e50 2023-07-06T02:23:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c0c4e90 con 0x7f255c0b5e50 2023-07-06T02:23:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f2552ffd700 1 -- 172.21.15.89:0/4081575532 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580139a0 con 0x7f255c0b5e50 2023-07-06T02:23:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.792+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c0c5450 con 0x7f255c0b5e50 2023-07-06T02:23:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.793+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c0c5610 con 0x7f255c0b5e50 2023-07-06T02:23:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.794+0000 7f2552ffd700 1 -- 172.21.15.89:0/4081575532 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2558013b00 con 0x7f255c0b5e50 2023-07-06T02:23:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.794+0000 7f2552ffd700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f254006ace0 0x7f254006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.794+0000 7f2552ffd700 1 -- 172.21.15.89:0/4081575532 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2558089080 con 0x7f255c0b5e50 2023-07-06T02:23:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.794+0000 7f2562598700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f254006ace0 0x7f254006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.795+0000 7f2562598700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f254006ace0 0x7f254006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f254c014040 tx=0x7f254c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.798+0000 7f2552ffd700 1 -- 172.21.15.89:0/4081575532 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2558053ed0 con 0x7f255c0b5e50 2023-07-06T02:23:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.988+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f255c0d03e0 con 0x7f255c0b5e50 2023-07-06T02:23:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.990+0000 7f2552ffd700 1 -- 172.21.15.89:0/4081575532 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f2558013db0 con 0x7f255c0b5e50 2023-07-06T02:23:35.992 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:35.992 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f254006ace0 msgr2=0x7f254006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f254006ace0 0x7f254006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f254c014040 tx=0x7f254c008030 comp rx=0 tx=0).stop 2023-07-06T02:23:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c0b5e50 msgr2=0x7f255c150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c0b5e50 0x7f255c150450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f255800e810 tx=0x7f255800eb20 comp rx=0 tx=0).stop 2023-07-06T02:23:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 shutdown_connections 2023-07-06T02:23:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255c15c610 0x7f255c0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255c070870 0x7f255c0af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f254006ace0 0x7f254006d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 --2- 172.21.15.89:0/4081575532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f255c0b5e50 0x7f255c150450 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 >> 172.21.15.89:0/4081575532 conn(0x7f255c0d0b00 msgr2=0x7f255c0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 shutdown_connections 2023-07-06T02:23:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:35.992+0000 7f25647fc700 1 -- 172.21.15.89:0/4081575532 wait complete. 2023-07-06T02:23:36.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:36.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:36.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.129+0000 7f447ba07700 1 Processor -- start 2023-07-06T02:23:36.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.130+0000 7f447ba07700 1 -- start start 2023-07-06T02:23:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.130+0000 7f447ba07700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f447415c690 0x7f44740cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.130+0000 7f447ba07700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f44740cbda0 0x7f44740b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.130+0000 7f447ba07700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44740b19d0 0x7f447406bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.130+0000 7f447ba07700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447415de50 con 0x7f447415c690 2023-07-06T02:23:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.130+0000 7f447ba07700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740cab50 con 0x7f44740cbda0 2023-07-06T02:23:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.130+0000 7f447ba07700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740b1e10 con 0x7f44740b19d0 2023-07-06T02:23:36.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f447415c690 0x7f44740cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f447415c690 0x7f44740cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33426/0 (socket says 172.21.15.89:33426) 2023-07-06T02:23:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 -- 172.21.15.89:0/3688489675 learned_addr learned my addr 172.21.15.89:0/3688489675 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:36.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 -- 172.21.15.89:0/3688489675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44740b19d0 msgr2=0x7f447406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 --2- 172.21.15.89:0/3688489675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44740b19d0 0x7f447406bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 -- 172.21.15.89:0/3688489675 >> v2:172.21.15.172:3300/0 conn(0x7f44740cbda0 msgr2=0x7f44740b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f4478fa2700 1 --2- 172.21.15.89:0/3688489675 >> v2:172.21.15.172:3300/0 conn(0x7f44740cbda0 0x7f44740b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 --2- 172.21.15.89:0/3688489675 >> v2:172.21.15.172:3300/0 conn(0x7f44740cbda0 0x7f44740b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 -- 172.21.15.89:0/3688489675 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447415db70 con 0x7f447415c690 2023-07-06T02:23:36.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f44797a3700 1 --2- 172.21.15.89:0/3688489675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f447415c690 0x7f44740cb860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4474070180 tx=0x7f446400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee77da0a436848d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f446bfff700 1 -- 172.21.15.89:0/3688489675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400dd30 con 0x7f447415c690 2023-07-06T02:23:36.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f446bfff700 1 -- 172.21.15.89:0/3688489675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446400de90 con 0x7f447415c690 2023-07-06T02:23:36.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.131+0000 7f446bfff700 1 -- 172.21.15.89:0/3688489675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464010690 con 0x7f447415c690 2023-07-06T02:23:36.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 -- 172.21.15.89:0/3688489675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f447415c690 msgr2=0x7f44740cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 --2- 172.21.15.89:0/3688489675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f447415c690 0x7f44740cb860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4474070180 tx=0x7f446400be70 comp rx=0 tx=0).stop 2023-07-06T02:23:36.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 -- 172.21.15.89:0/3688489675 shutdown_connections 2023-07-06T02:23:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 --2- 172.21.15.89:0/3688489675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44740b19d0 0x7f447406bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 --2- 172.21.15.89:0/3688489675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f447415c690 0x7f44740cb860 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 --2- 172.21.15.89:0/3688489675 >> v2:172.21.15.172:3300/0 conn(0x7f44740cbda0 0x7f44740b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 -- 172.21.15.89:0/3688489675 >> 172.21.15.89:0/3688489675 conn(0x7f44740cc490 msgr2=0x7f44740cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 -- 172.21.15.89:0/3688489675 shutdown_connections 2023-07-06T02:23:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.132+0000 7f447ba07700 1 -- 172.21.15.89:0/3688489675 wait complete. 2023-07-06T02:23:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.133+0000 7f447ba07700 1 Processor -- start 2023-07-06T02:23:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.133+0000 7f447ba07700 1 -- start start 2023-07-06T02:23:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f447ba07700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44740b19d0 0x7f44740b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f447ba07700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44740cbda0 0x7f44740c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f447ba07700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f447415c690 0x7f4474150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f447ba07700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44741f6740 con 0x7f44740cbda0 2023-07-06T02:23:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f447ba07700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740c42c0 con 0x7f44740b19d0 2023-07-06T02:23:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f447ba07700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740c45a0 con 0x7f447415c690 2023-07-06T02:23:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44740cbda0 0x7f44740c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44740cbda0 0x7f44740c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33436/0 (socket says 172.21.15.89:33436) 2023-07-06T02:23:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 -- 172.21.15.89:0/2580723970 learned_addr learned my addr 172.21.15.89:0/2580723970 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4479fa4700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f447415c690 0x7f4474150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f44797a3700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44740b19d0 0x7f44740b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 -- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f447415c690 msgr2=0x7f4474150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f447415c690 0x7f4474150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 -- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44740b19d0 msgr2=0x7f44740b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44740b19d0 0x7f44740b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4478fa2700 1 -- 172.21.15.89:0/2580723970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446400b7f0 con 0x7f44740cbda0 2023-07-06T02:23:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f4479fa4700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f447415c690 0x7f4474150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.134+0000 7f44797a3700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44740b19d0 0x7f44740b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.135+0000 7f4478fa2700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44740cbda0 0x7f44740c8ac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f447000c7f0 tx=0x7f447000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.135+0000 7f4469ffb700 1 -- 172.21.15.89:0/2580723970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470011800 con 0x7f44740cbda0 2023-07-06T02:23:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.135+0000 7f4469ffb700 1 -- 172.21.15.89:0/2580723970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470011960 con 0x7f44740cbda0 2023-07-06T02:23:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.135+0000 7f4469ffb700 1 -- 172.21.15.89:0/2580723970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470011cf0 con 0x7f44740cbda0 2023-07-06T02:23:36.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.135+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44740c4880 con 0x7f44740cbda0 2023-07-06T02:23:36.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.135+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44740c4e40 con 0x7f44740cbda0 2023-07-06T02:23:36.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.136+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44740c5030 con 0x7f44740cbda0 2023-07-06T02:23:36.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.136+0000 7f4469ffb700 1 -- 172.21.15.89:0/2580723970 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4470018360 con 0x7f44740cbda0 2023-07-06T02:23:36.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.136+0000 7f4469ffb700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f445806ace0 0x7f445806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.137+0000 7f4469ffb700 1 -- 172.21.15.89:0/2580723970 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4470089700 con 0x7f44740cbda0 2023-07-06T02:23:36.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.137+0000 7f44797a3700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f445806ace0 0x7f445806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.137+0000 7f44797a3700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f445806ace0 0x7f445806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f446400beb0 tx=0x7f4464012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.141+0000 7f4469ffb700 1 -- 172.21.15.89:0/2580723970 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4470054550 con 0x7f44740cbda0 2023-07-06T02:23:36.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.327+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4474074270 con 0x7f44740cbda0 2023-07-06T02:23:36.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.328+0000 7f4469ffb700 1 -- 172.21.15.89:0/2580723970 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4470057b70 con 0x7f44740cbda0 2023-07-06T02:23:36.330 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f445806ace0 msgr2=0x7f445806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f445806ace0 0x7f445806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f446400beb0 tx=0x7f4464012000 comp rx=0 tx=0).stop 2023-07-06T02:23:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44740cbda0 msgr2=0x7f44740c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44740cbda0 0x7f44740c8ac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f447000c7f0 tx=0x7f447000cbb0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 shutdown_connections 2023-07-06T02:23:36.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f447415c690 0x7f4474150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44740b19d0 0x7f44740b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f445806ace0 0x7f445806d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 --2- 172.21.15.89:0/2580723970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44740cbda0 0x7f44740c8ac0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 >> 172.21.15.89:0/2580723970 conn(0x7f44740cc490 msgr2=0x7f447415ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 shutdown_connections 2023-07-06T02:23:36.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.330+0000 7f447ba07700 1 -- 172.21.15.89:0/2580723970 wait complete. 2023-07-06T02:23:36.343 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:36.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.465+0000 7faa48eeb700 1 Processor -- start 2023-07-06T02:23:36.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.466+0000 7faa48eeb700 1 -- start start 2023-07-06T02:23:36.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.466+0000 7faa48eeb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 0x7faa44070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.466+0000 7faa48eeb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faa44070870 0x7faa440cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.466+0000 7faa48eeb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440cb150 con 0x7faa4415c610 2023-07-06T02:23:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa48eeb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa440d05e0 con 0x7faa44070870 2023-07-06T02:23:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa48eeb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa44070cf0 con 0x7faa440b5e50 2023-07-06T02:23:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4259c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faa4415c610 0x7faa440cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58342/0 (socket says 172.21.15.89:58342) 2023-07-06T02:23:36.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4259c700 1 -- 172.21.15.89:0/2579136121 learned_addr learned my addr 172.21.15.89:0/2579136121 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa41d9b700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 0x7faa44070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597226119 0 0) 0x7faa440cb150 con 0x7faa4415c610 2023-07-06T02:23:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa300035b0 con 0x7faa4415c610 2023-07-06T02:23:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa42d9d700 1 --2- 172.21.15.89:0/2579136121 >> v2:172.21.15.172:3300/0 conn(0x7faa44070870 0x7faa440cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1626702479 0 0) 0x7faa300035b0 con 0x7faa4415c610 2023-07-06T02:23:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 msgr2=0x7faa44070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:36.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 0x7faa44070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 >> v2:172.21.15.172:3300/0 conn(0x7faa44070870 msgr2=0x7faa440cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 --2- 172.21.15.89:0/2579136121 >> v2:172.21.15.172:3300/0 conn(0x7faa44070870 0x7faa440cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440cb430 con 0x7faa4415c610 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7faa2c001670 con 0x7faa4415c610 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 >> v1:172.21.15.89:6789/0 conn(0x7faa4415c610 legacy=0x7faa440cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa30003a00 0x7faa30003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 0x7faa30004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa30005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440d05e0 con 0x7faa30004d90 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44070cf0 con 0x7faa30003a00 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3000ba10 con 0x7faa440b5e50 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa42d9d700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa30005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa41d9b700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 0x7faa30004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.467+0000 7faa4259c700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa30003a00 0x7faa30003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa42d9d700 1 -- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 msgr2=0x7faa30004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:36.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa42d9d700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440b5e50 0x7faa30004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa42d9d700 1 -- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa30003a00 msgr2=0x7faa30003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa42d9d700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa30003a00 0x7faa30003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa42d9d700 1 -- 172.21.15.89:0/2579136121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300035b0 con 0x7faa30004d90 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa42d9d700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa30005200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009fc0 tx=0x7faa30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5db8d632b000f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00ebd0 con 0x7faa30004d90 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c00f040 con 0x7faa30004d90 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa4159a700 1 -- 172.21.15.89:0/2579136121 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c014480 con 0x7faa30004d90 2023-07-06T02:23:36.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa48eeb700 1 -- 172.21.15.89:0/2579136121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 msgr2=0x7faa30005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa48eeb700 1 --2- 172.21.15.89:0/2579136121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa30005200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009fc0 tx=0x7faa30005880 comp rx=0 tx=0).stop 2023-07-06T02:23:36.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.468+0000 7faa42d9d700 1 -- 172.21.15.89:0/2579136121 reap_dead start 2023-07-06T02:23:36.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.469+0000 7faa48eeb700 1 -- 172.21.15.89:0/2579136121 shutdown_connections 2023-07-06T02:23:36.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.469+0000 7faa48eeb700 1 -- 172.21.15.89:0/2579136121 >> 172.21.15.89:0/2579136121 conn(0x7faa440d0b00 msgr2=0x7faa440d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:36.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.469+0000 7faa48eeb700 1 -- 172.21.15.89:0/2579136121 shutdown_connections 2023-07-06T02:23:36.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.469+0000 7faa48eeb700 1 -- 172.21.15.89:0/2579136121 wait complete. 2023-07-06T02:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.470+0000 7faa48eeb700 1 Processor -- start 2023-07-06T02:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.470+0000 7faa48eeb700 1 -- start start 2023-07-06T02:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa44153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440c6090 0x7faa44149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa44153aa0 0x7faa44153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44154480 con 0x7faa30004d90 2023-07-06T02:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440c6590 con 0x7faa44153aa0 2023-07-06T02:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440c6870 con 0x7faa440c6090 2023-07-06T02:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa44153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa44153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33452/0 (socket says 172.21.15.89:33452) 2023-07-06T02:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 -- 172.21.15.89:0/2918373103 learned_addr learned my addr 172.21.15.89:0/2918373103 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa42d9d700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa44153aa0 0x7faa44153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa41d9b700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440c6090 0x7faa44149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 -- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440c6090 msgr2=0x7faa44149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440c6090 0x7faa44149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 -- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa44153aa0 msgr2=0x7faa44153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa44153aa0 0x7faa44153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 -- 172.21.15.89:0/2918373103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440c6b50 con 0x7faa30004d90 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa42d9d700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa44153aa0 0x7faa44153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa4259c700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa44153520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faa2c007310 tx=0x7faa2c0076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa3affd700 1 -- 172.21.15.89:0/2918373103 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0077f0 con 0x7faa30004d90 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa3affd700 1 -- 172.21.15.89:0/2918373103 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c00f8d0 con 0x7faa30004d90 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa440c6e30 con 0x7faa30004d90 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa3affd700 1 -- 172.21.15.89:0/2918373103 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00fc80 con 0x7faa30004d90 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.471+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa44153180 con 0x7faa30004d90 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.472+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa440c7020 con 0x7faa30004d90 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.473+0000 7faa3affd700 1 -- 172.21.15.89:0/2918373103 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa2c01d460 con 0x7faa30004d90 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.473+0000 7faa3affd700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7faa2406ace0 0x7faa2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.473+0000 7faa3affd700 1 -- 172.21.15.89:0/2918373103 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faa2c089870 con 0x7faa30004d90 2023-07-06T02:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.473+0000 7faa41d9b700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7faa2406ace0 0x7faa2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.473+0000 7faa41d9b700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7faa2406ace0 0x7faa2406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa4415fba0 tx=0x7faa34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.477+0000 7faa3affd700 1 -- 172.21.15.89:0/2918373103 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faa2c054630 con 0x7faa30004d90 2023-07-06T02:23:36.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.661+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa44074270 con 0x7faa30004d90 2023-07-06T02:23:36.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.661+0000 7faa3affd700 1 -- 172.21.15.89:0/2918373103 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa2c057c50 con 0x7faa30004d90 2023-07-06T02:23:36.664 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:36.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7faa2406ace0 msgr2=0x7faa2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7faa2406ace0 0x7faa2406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa4415fba0 tx=0x7faa34008040 comp rx=0 tx=0).stop 2023-07-06T02:23:36.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 msgr2=0x7faa44153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa44153520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faa2c007310 tx=0x7faa2c0076d0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 shutdown_connections 2023-07-06T02:23:36.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa44153aa0 0x7faa44153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7faa2406ace0 0x7faa2406d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa30004d90 0x7faa44153520 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.663+0000 7faa48eeb700 1 --2- 172.21.15.89:0/2918373103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa440c6090 0x7faa44149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.664+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 >> 172.21.15.89:0/2918373103 conn(0x7faa440d0b00 msgr2=0x7faa440b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:36.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.664+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 shutdown_connections 2023-07-06T02:23:36.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.664+0000 7faa48eeb700 1 -- 172.21.15.89:0/2918373103 wait complete. 2023-07-06T02:23:36.677 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.797+0000 7f341bffc700 1 Processor -- start 2023-07-06T02:23:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.797+0000 7f341bffc700 1 -- start start 2023-07-06T02:23:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f341bffc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f341406dc10 0x7f341415a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f341bffc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341415adb0 0x7f3414157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f341bffc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140cd350 con 0x7f34140705a0 2023-07-06T02:23:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f341bffc700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341406e110 con 0x7f341406dc10 2023-07-06T02:23:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f341bffc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341406e3f0 con 0x7f341415adb0 2023-07-06T02:23:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f3419597700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f341406dc10 0x7f341415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f3419d98700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f34140705a0 0x7f34140ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58362/0 (socket says 172.21.15.89:58362) 2023-07-06T02:23:36.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f3419d98700 1 -- 172.21.15.89:0/3175521469 learned_addr learned my addr 172.21.15.89:0/3175521469 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:36.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f341a599700 1 --2- 172.21.15.89:0/3175521469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341415adb0 0x7f3414157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.798+0000 7f3418d96700 1 -- 172.21.15.89:0/3175521469 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604554299 0 0) 0x7f34140cd350 con 0x7f34140705a0 2023-07-06T02:23:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3418d96700 1 -- 172.21.15.89:0/3175521469 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34000035b0 con 0x7f34140705a0 2023-07-06T02:23:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3419597700 1 -- 172.21.15.89:0/3175521469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341415adb0 msgr2=0x7f3414157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3419597700 1 --2- 172.21.15.89:0/3175521469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341415adb0 0x7f3414157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3419597700 1 -- 172.21.15.89:0/3175521469 >> v1:172.21.15.89:6789/0 conn(0x7f34140705a0 legacy=0x7f34140ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3419597700 1 -- 172.21.15.89:0/3175521469 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341406e6d0 con 0x7f341406dc10 2023-07-06T02:23:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3419597700 1 --2- 172.21.15.89:0/3175521469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f341406dc10 0x7f341415a870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3410009870 tx=0x7f3410009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fd210945152771 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3418d96700 1 -- 172.21.15.89:0/3175521469 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011830 con 0x7f341406dc10 2023-07-06T02:23:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3418d96700 1 -- 172.21.15.89:0/3175521469 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410011990 con 0x7f341406dc10 2023-07-06T02:23:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f3418d96700 1 -- 172.21.15.89:0/3175521469 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011d20 con 0x7f341406dc10 2023-07-06T02:23:36.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f341bffc700 1 -- 172.21.15.89:0/3175521469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f341406dc10 msgr2=0x7f341415a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f341bffc700 1 --2- 172.21.15.89:0/3175521469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f341406dc10 0x7f341415a870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3410009870 tx=0x7f3410009b80 comp rx=0 tx=0).stop 2023-07-06T02:23:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f341bffc700 1 -- 172.21.15.89:0/3175521469 shutdown_connections 2023-07-06T02:23:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f341bffc700 1 --2- 172.21.15.89:0/3175521469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341415adb0 0x7f3414157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f341bffc700 1 --2- 172.21.15.89:0/3175521469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f341406dc10 0x7f341415a870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.799+0000 7f341bffc700 1 -- 172.21.15.89:0/3175521469 >> 172.21.15.89:0/3175521469 conn(0x7f34140d0b00 msgr2=0x7f34140d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:36.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.800+0000 7f341bffc700 1 -- 172.21.15.89:0/3175521469 shutdown_connections 2023-07-06T02:23:36.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.800+0000 7f341bffc700 1 -- 172.21.15.89:0/3175521469 wait complete. 2023-07-06T02:23:36.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.801+0000 7f341bffc700 1 Processor -- start 2023-07-06T02:23:36.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.801+0000 7f341bffc700 1 -- start start 2023-07-06T02:23:36.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.801+0000 7f341bffc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341406dc10 0x7f34140a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.801+0000 7f341bffc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34140705a0 0x7f34141ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.801+0000 7f341bffc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341415adb0 0x7f34140b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.801+0000 7f341bffc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414148860 con 0x7f34140705a0 2023-07-06T02:23:36.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f341bffc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140b53f0 con 0x7f341415adb0 2023-07-06T02:23:36.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f341bffc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34140b56d0 con 0x7f341406dc10 2023-07-06T02:23:36.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34140705a0 0x7f34141ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34140705a0 0x7f34141ef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33468/0 (socket says 172.21.15.89:33468) 2023-07-06T02:23:36.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 -- 172.21.15.89:0/1403866653 learned_addr learned my addr 172.21.15.89:0/1403866653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:36.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419d98700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341406dc10 0x7f34140a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f341a599700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341415adb0 0x7f34140b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 -- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341406dc10 msgr2=0x7f34140a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341406dc10 0x7f34140a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 -- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341415adb0 msgr2=0x7f34140b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:36.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341415adb0 0x7f34140b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:36.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 -- 172.21.15.89:0/1403866653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34100094a0 con 0x7f34140705a0 2023-07-06T02:23:36.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f3419597700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34140705a0 0x7f34141ef8c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3410000c00 tx=0x7f3410004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f340a7fc700 1 -- 172.21.15.89:0/1403866653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100036a0 con 0x7f34140705a0 2023-07-06T02:23:36.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f340a7fc700 1 -- 172.21.15.89:0/1403866653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341001f070 con 0x7f34140705a0 2023-07-06T02:23:36.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34140b59b0 con 0x7f34140705a0 2023-07-06T02:23:36.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f340a7fc700 1 -- 172.21.15.89:0/1403866653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410003a60 con 0x7f34140705a0 2023-07-06T02:23:36.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.802+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34141efe00 con 0x7f34140705a0 2023-07-06T02:23:36.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.803+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3414074270 con 0x7f34140705a0 2023-07-06T02:23:36.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.804+0000 7f340a7fc700 1 -- 172.21.15.89:0/1403866653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3410003bc0 con 0x7f34140705a0 2023-07-06T02:23:36.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.804+0000 7f340a7fc700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f33f806ace0 0x7f33f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:36.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.804+0000 7f3419d98700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f33f806ace0 0x7f33f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:36.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.804+0000 7f340a7fc700 1 -- 172.21.15.89:0/1403866653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3410021070 con 0x7f34140705a0 2023-07-06T02:23:36.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.805+0000 7f3419d98700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f33f806ace0 0x7f33f806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3404007fd0 tx=0x7f3404007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:36.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.809+0000 7f340a7fc700 1 -- 172.21.15.89:0/1403866653 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3410055cb0 con 0x7f34140705a0 2023-07-06T02:23:36.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.994+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34140c3f80 con 0x7f34140705a0 2023-07-06T02:23:36.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.995+0000 7f340a7fc700 1 -- 172.21.15.89:0/1403866653 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34100150a0 con 0x7f34140705a0 2023-07-06T02:23:36.997 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:36.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f33f806ace0 msgr2=0x7f33f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f33f806ace0 0x7f33f806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3404007fd0 tx=0x7f3404007dc0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34140705a0 msgr2=0x7f34141ef8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34140705a0 0x7f34141ef8c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3410000c00 tx=0x7f3410004750 comp rx=0 tx=0).stop 2023-07-06T02:23:37.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 shutdown_connections 2023-07-06T02:23:37.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341415adb0 0x7f34140b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f341406dc10 0x7f34140a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f33f806ace0 0x7f33f806d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 --2- 172.21.15.89:0/1403866653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34140705a0 0x7f34141ef8c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 >> 172.21.15.89:0/1403866653 conn(0x7f34140d0b00 msgr2=0x7f34140ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:37.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 shutdown_connections 2023-07-06T02:23:37.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:36.997+0000 7f341bffc700 1 -- 172.21.15.89:0/1403866653 wait complete. 2023-07-06T02:23:37.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:37.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.132+0000 7f12b1de9700 1 Processor -- start 2023-07-06T02:23:37.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.132+0000 7f12b1de9700 1 -- start start 2023-07-06T02:23:37.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12b1de9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0705a0 0x7f12ac0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12b1de9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12ac0cd180 0x7f12ac15a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12b1de9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac06e110 con 0x7f12ac06dc10 2023-07-06T02:23:37.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12b1de9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac06e3f0 con 0x7f12ac0cd180 2023-07-06T02:23:37.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12b1de9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac06e6d0 con 0x7f12ac0705a0 2023-07-06T02:23:37.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12ab7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0705a0 0x7f12ac0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12aaffd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12ac0cd180 0x7f12ac15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12aaffd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12ac0cd180 0x7f12ac15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:60464/0 (socket says 172.21.15.89:60464) 2023-07-06T02:23:37.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12ab7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0705a0 0x7f12ac0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:56682/0 (socket says 172.21.15.89:56682) 2023-07-06T02:23:37.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.133+0000 7f12aaffd700 1 -- 172.21.15.89:0/399700980 learned_addr learned my addr 172.21.15.89:0/399700980 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:37.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aaffd700 1 -- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0705a0 msgr2=0x7f12ac0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:37.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aaffd700 1 --2- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0705a0 0x7f12ac0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aaffd700 1 -- 172.21.15.89:0/399700980 >> v1:172.21.15.172:6789/0 conn(0x7f12ac06dc10 legacy=0x7f12ac157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aaffd700 1 -- 172.21.15.89:0/399700980 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac06e9b0 con 0x7f12ac0cd180 2023-07-06T02:23:37.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12ab7fe700 1 --2- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0705a0 0x7f12ac0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:37.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aaffd700 1 --2- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 0x7f12ac15a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f129c0098f0 tx=0x7f129c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a58f8b52513b2c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aa7fc700 1 -- 172.21.15.89:0/399700980 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c011830 con 0x7f12ac0cd180 2023-07-06T02:23:37.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aa7fc700 1 -- 172.21.15.89:0/399700980 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c011990 con 0x7f12ac0cd180 2023-07-06T02:23:37.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12aa7fc700 1 -- 172.21.15.89:0/399700980 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c011d20 con 0x7f12ac0cd180 2023-07-06T02:23:37.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12b1de9700 1 -- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 msgr2=0x7f12ac15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12b1de9700 1 --2- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 0x7f12ac15a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f129c0098f0 tx=0x7f129c009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:37.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12b1de9700 1 -- 172.21.15.89:0/399700980 shutdown_connections 2023-07-06T02:23:37.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12b1de9700 1 --2- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0705a0 0x7f12ac0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.134+0000 7f12b1de9700 1 --2- 172.21.15.89:0/399700980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 0x7f12ac15a870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.135+0000 7f12b1de9700 1 -- 172.21.15.89:0/399700980 >> 172.21.15.89:0/399700980 conn(0x7f12ac0d0b00 msgr2=0x7f12ac0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:37.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.135+0000 7f12b1de9700 1 -- 172.21.15.89:0/399700980 shutdown_connections 2023-07-06T02:23:37.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.135+0000 7f12b1de9700 1 -- 172.21.15.89:0/399700980 wait complete. 2023-07-06T02:23:37.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.136+0000 7f12b1de9700 1 Processor -- start 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.136+0000 7f12b1de9700 1 -- start start 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12b1de9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac06dc10 0x7f12ac148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12b1de9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ac0705a0 0x7f12ac1f0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12b1de9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 0x7f12ac152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12b1de9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1532f0 con 0x7f12ac0705a0 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12b1de9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac156070 con 0x7f12ac0cd180 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12b1de9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1f0880 con 0x7f12ac06dc10 2023-07-06T02:23:37.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ac0705a0 0x7f12ac1f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ac0705a0 0x7f12ac1f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33482/0 (socket says 172.21.15.89:33482) 2023-07-06T02:23:37.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 -- 172.21.15.89:0/2616746007 learned_addr learned my addr 172.21.15.89:0/2616746007 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:37.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12ab7fe700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac06dc10 0x7f12ac148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12abfff700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 0x7f12ac152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 -- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac06dc10 msgr2=0x7f12ac148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac06dc10 0x7f12ac148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 -- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 msgr2=0x7f12ac152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 0x7f12ac152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 -- 172.21.15.89:0/2616746007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00094a0 con 0x7f12ac0705a0 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12aaffd700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ac0705a0 0x7f12ac1f0200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f129c009c40 tx=0x7f129c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f1293fff700 1 -- 172.21.15.89:0/2616746007 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0036a0 con 0x7f12ac0705a0 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f1293fff700 1 -- 172.21.15.89:0/2616746007 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c01e070 con 0x7f12ac0705a0 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c009550 con 0x7f12ac0705a0 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.137+0000 7f1293fff700 1 -- 172.21.15.89:0/2616746007 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c003a60 con 0x7f12ac0705a0 2023-07-06T02:23:37.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.138+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac1f0e40 con 0x7f12ac0705a0 2023-07-06T02:23:37.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.138+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac074270 con 0x7f12ac0705a0 2023-07-06T02:23:37.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.139+0000 7f1293fff700 1 -- 172.21.15.89:0/2616746007 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f129c003bc0 con 0x7f12ac0705a0 2023-07-06T02:23:37.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.139+0000 7f1293fff700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f129806ace0 0x7f129806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.140+0000 7f1293fff700 1 -- 172.21.15.89:0/2616746007 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f129c08a650 con 0x7f12ac0705a0 2023-07-06T02:23:37.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.140+0000 7f12ab7fe700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f129806ace0 0x7f129806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.140+0000 7f12ab7fe700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f129806ace0 0x7f129806d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009d00 tx=0x7f12a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.144+0000 7f1293fff700 1 -- 172.21.15.89:0/2616746007 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f129c0554a0 con 0x7f12ac0705a0 2023-07-06T02:23:37.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.331+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12ac0cfea0 con 0x7f12ac0705a0 2023-07-06T02:23:37.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.332+0000 7f1293fff700 1 -- 172.21.15.89:0/2616746007 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f129c058ac0 con 0x7f12ac0705a0 2023-07-06T02:23:37.334 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:37.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f129806ace0 msgr2=0x7f129806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f129806ace0 0x7f129806d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009d00 tx=0x7f12a0008040 comp rx=0 tx=0).stop 2023-07-06T02:23:37.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ac0705a0 msgr2=0x7f12ac1f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ac0705a0 0x7f12ac1f0200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f129c009c40 tx=0x7f129c004750 comp rx=0 tx=0).stop 2023-07-06T02:23:37.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 shutdown_connections 2023-07-06T02:23:37.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac0cd180 0x7f12ac152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac06dc10 0x7f12ac148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f129806ace0 0x7f129806d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.333+0000 7f12b1de9700 1 --2- 172.21.15.89:0/2616746007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ac0705a0 0x7f12ac1f0200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.334+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 >> 172.21.15.89:0/2616746007 conn(0x7f12ac0d0b00 msgr2=0x7f12ac0ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:37.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.334+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 shutdown_connections 2023-07-06T02:23:37.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.334+0000 7f12b1de9700 1 -- 172.21.15.89:0/2616746007 wait complete. 2023-07-06T02:23:37.347 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:37.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.471+0000 7f9074cfe700 1 Processor -- start 2023-07-06T02:23:37.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.471+0000 7f9074cfe700 1 -- start start 2023-07-06T02:23:37.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.472+0000 7f9074cfe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f907015c690 0x7f90700cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.472+0000 7f9074cfe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f90700cbda0 0x7f90700b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.472+0000 7f9074cfe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907006bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.472+0000 7f9074cfe700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907015de50 con 0x7f907015c690 2023-07-06T02:23:37.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.472+0000 7f9074cfe700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90700cab50 con 0x7f90700cbda0 2023-07-06T02:23:37.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.472+0000 7f9074cfe700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90700b1e10 con 0x7f90700b19d0 2023-07-06T02:23:37.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f907015c690 0x7f90700cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f907015c690 0x7f90700cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33488/0 (socket says 172.21.15.89:33488) 2023-07-06T02:23:37.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 -- 172.21.15.89:0/2504255815 learned_addr learned my addr 172.21.15.89:0/2504255815 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:37.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906ed9d700 1 --2- 172.21.15.89:0/2504255815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907006bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906dd9b700 1 --2- 172.21.15.89:0/2504255815 >> v2:172.21.15.172:3300/0 conn(0x7f90700cbda0 0x7f90700b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 -- 172.21.15.89:0/2504255815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 msgr2=0x7f907006bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 --2- 172.21.15.89:0/2504255815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907006bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 -- 172.21.15.89:0/2504255815 >> v2:172.21.15.172:3300/0 conn(0x7f90700cbda0 msgr2=0x7f90700b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 --2- 172.21.15.89:0/2504255815 >> v2:172.21.15.172:3300/0 conn(0x7f90700cbda0 0x7f90700b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 -- 172.21.15.89:0/2504255815 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907015db70 con 0x7f907015c690 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906ed9d700 1 --2- 172.21.15.89:0/2504255815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907006bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:37.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906dd9b700 1 --2- 172.21.15.89:0/2504255815 >> v2:172.21.15.172:3300/0 conn(0x7f90700cbda0 0x7f90700b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:37.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906e59c700 1 --2- 172.21.15.89:0/2504255815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f90700cb860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9070070180 tx=0x7f906400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b0ade91a142bb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906d59a700 1 -- 172.21.15.89:0/2504255815 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906400dd30 con 0x7f907015c690 2023-07-06T02:23:37.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906d59a700 1 -- 172.21.15.89:0/2504255815 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906400de90 con 0x7f907015c690 2023-07-06T02:23:37.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.473+0000 7f906d59a700 1 -- 172.21.15.89:0/2504255815 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064010690 con 0x7f907015c690 2023-07-06T02:23:37.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 -- 172.21.15.89:0/2504255815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 msgr2=0x7f90700cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 --2- 172.21.15.89:0/2504255815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f90700cb860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9070070180 tx=0x7f906400be70 comp rx=0 tx=0).stop 2023-07-06T02:23:37.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 -- 172.21.15.89:0/2504255815 shutdown_connections 2023-07-06T02:23:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 --2- 172.21.15.89:0/2504255815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907006bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 --2- 172.21.15.89:0/2504255815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f90700cb860 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 --2- 172.21.15.89:0/2504255815 >> v2:172.21.15.172:3300/0 conn(0x7f90700cbda0 0x7f90700b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 -- 172.21.15.89:0/2504255815 >> 172.21.15.89:0/2504255815 conn(0x7f90700cc490 msgr2=0x7f90700cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 -- 172.21.15.89:0/2504255815 shutdown_connections 2023-07-06T02:23:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.474+0000 7f9074cfe700 1 -- 172.21.15.89:0/2504255815 wait complete. 2023-07-06T02:23:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.475+0000 7f9074cfe700 1 Processor -- start 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.475+0000 7f9074cfe700 1 -- start start 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f9074cfe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907014fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f9074cfe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90700cbda0 0x7f90701506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f9074cfe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f907014f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f9074cfe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90701502d0 con 0x7f907015c690 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f9074cfe700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700b2860 con 0x7f90700cbda0 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f9074cfe700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90700c8050 con 0x7f90700b19d0 2023-07-06T02:23:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f907014f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f907014f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33498/0 (socket says 172.21.15.89:33498) 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 -- 172.21.15.89:0/3474157581 learned_addr learned my addr 172.21.15.89:0/3474157581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 -- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 msgr2=0x7f907014fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907014fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 -- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90700cbda0 msgr2=0x7f90701506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906dd9b700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90700cbda0 0x7f90701506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90700cbda0 0x7f90701506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 -- 172.21.15.89:0/3474157581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906400b7f0 con 0x7f907015c690 2023-07-06T02:23:37.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906dd9b700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90700cbda0 0x7f90701506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:37.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.476+0000 7f906ed9d700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f907014f1c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9068007df0 tx=0x7f906800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.477+0000 7f905effd700 1 -- 172.21.15.89:0/3474157581 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800ddd0 con 0x7f907015c690 2023-07-06T02:23:37.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.477+0000 7f905effd700 1 -- 172.21.15.89:0/3474157581 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800f040 con 0x7f907015c690 2023-07-06T02:23:37.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.477+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9070150d80 con 0x7f907015c690 2023-07-06T02:23:37.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.477+0000 7f905effd700 1 -- 172.21.15.89:0/3474157581 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068003c60 con 0x7f907015c690 2023-07-06T02:23:37.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.477+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9070151340 con 0x7f907015c690 2023-07-06T02:23:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.478+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9070074270 con 0x7f907015c690 2023-07-06T02:23:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.478+0000 7f905effd700 1 -- 172.21.15.89:0/3474157581 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9068004030 con 0x7f907015c690 2023-07-06T02:23:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.478+0000 7f905effd700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f905006ace0 0x7f905006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.478+0000 7f905effd700 1 -- 172.21.15.89:0/3474157581 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9068089980 con 0x7f907015c690 2023-07-06T02:23:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.478+0000 7f906e59c700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f905006ace0 0x7f905006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.479+0000 7f906e59c700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f905006ace0 0x7f905006d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9064000c00 tx=0x7f906400bf90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.483+0000 7f905effd700 1 -- 172.21.15.89:0/3474157581 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9068054730 con 0x7f907015c690 2023-07-06T02:23:37.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.668+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f907015a450 con 0x7f907015c690 2023-07-06T02:23:37.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.668+0000 7f905effd700 1 -- 172.21.15.89:0/3474157581 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9068057d50 con 0x7f907015c690 2023-07-06T02:23:37.670 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f905006ace0 msgr2=0x7f905006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f905006ace0 0x7f905006d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9064000c00 tx=0x7f906400bf90 comp rx=0 tx=0).stop 2023-07-06T02:23:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 msgr2=0x7f907014f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f907014f1c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9068007df0 tx=0x7f906800d3b0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 shutdown_connections 2023-07-06T02:23:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f905006ace0 0x7f905006d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907015c690 0x7f907014f1c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90700b19d0 0x7f907014fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 --2- 172.21.15.89:0/3474157581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90700cbda0 0x7f90701506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 >> 172.21.15.89:0/3474157581 conn(0x7f90700cc490 msgr2=0x7f90700cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 shutdown_connections 2023-07-06T02:23:37.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.670+0000 7f9074cfe700 1 -- 172.21.15.89:0/3474157581 wait complete. 2023-07-06T02:23:37.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.802+0000 7fbdcc88f700 1 Processor -- start 2023-07-06T02:23:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.802+0000 7fbdcc88f700 1 -- start start 2023-07-06T02:23:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdcc88f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdcc88f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbdc8160bb0 0x7fbdc80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdcc88f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbdc81600b0 0x7fbdc80b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdcc88f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80cb3d0 con 0x7fbdc8160bb0 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdcc88f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc8156e80 con 0x7fbdc81600b0 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdcc88f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80ca370 con 0x7fbdc8156ab0 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbdc8160bb0 0x7fbdc80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbdc8160bb0 0x7fbdc80b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33502/0 (socket says 172.21.15.89:33502) 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/2405837510 learned_addr learned my addr 172.21.15.89:0/2405837510 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc6d9d700 1 --2- 172.21.15.89:0/2405837510 >> v2:172.21.15.172:3300/0 conn(0x7fbdc81600b0 0x7fbdc80b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc659c700 1 --2- 172.21.15.89:0/2405837510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/2405837510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 msgr2=0x7fbdc8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:37.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 --2- 172.21.15.89:0/2405837510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/2405837510 >> v2:172.21.15.172:3300/0 conn(0x7fbdc81600b0 msgr2=0x7fbdc80b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:37.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 --2- 172.21.15.89:0/2405837510 >> v2:172.21.15.172:3300/0 conn(0x7fbdc81600b0 0x7fbdc80b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/2405837510 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80ca610 con 0x7fbdc8160bb0 2023-07-06T02:23:37.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.803+0000 7fbdc659c700 1 --2- 172.21.15.89:0/2405837510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdc5d9b700 1 --2- 172.21.15.89:0/2405837510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc8160bb0 0x7fbdc80b1490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbdc815fef0 tx=0x7fbdb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4ee538221c99c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdc559a700 1 -- 172.21.15.89:0/2405837510 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800b870 con 0x7fbdc8160bb0 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdc559a700 1 -- 172.21.15.89:0/2405837510 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb800b9d0 con 0x7fbdc8160bb0 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdc559a700 1 -- 172.21.15.89:0/2405837510 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800bd60 con 0x7fbdc8160bb0 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 -- 172.21.15.89:0/2405837510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc8160bb0 msgr2=0x7fbdc80b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/2405837510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc8160bb0 0x7fbdc80b1490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbdc815fef0 tx=0x7fbdb8009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 -- 172.21.15.89:0/2405837510 shutdown_connections 2023-07-06T02:23:37.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/2405837510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/2405837510 >> v2:172.21.15.172:3300/0 conn(0x7fbdc81600b0 0x7fbdc80b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/2405837510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc8160bb0 0x7fbdc80b1490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 -- 172.21.15.89:0/2405837510 >> 172.21.15.89:0/2405837510 conn(0x7fbdc80cc490 msgr2=0x7fbdc80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:37.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 -- 172.21.15.89:0/2405837510 shutdown_connections 2023-07-06T02:23:37.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.804+0000 7fbdcc88f700 1 -- 172.21.15.89:0/2405837510 wait complete. 2023-07-06T02:23:37.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 Processor -- start 2023-07-06T02:23:37.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 -- start start 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc81600b0 0x7fbdc80c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc80acc30 0x7fbdc815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80addb0 con 0x7fbdc81600b0 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80ad130 con 0x7fbdc80acc30 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdcc88f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80ad410 con 0x7fbdc8156ab0 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.806+0000 7fbdc5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc81600b0 0x7fbdc80c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc81600b0 0x7fbdc80c4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33510/0 (socket says 172.21.15.89:33510) 2023-07-06T02:23:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/674249657 learned_addr learned my addr 172.21.15.89:0/674249657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc6d9d700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc80acc30 0x7fbdc815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc659c700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 msgr2=0x7fbdc8155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc80acc30 msgr2=0x7fbdc815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc80acc30 0x7fbdc815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 -- 172.21.15.89:0/674249657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8009550 con 0x7fbdc81600b0 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdc5d9b700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc81600b0 0x7fbdc80c4f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8000c00 tx=0x7fbdb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdb6ffd700 1 -- 172.21.15.89:0/674249657 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8004520 con 0x7fbdc81600b0 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdb6ffd700 1 -- 172.21.15.89:0/674249657 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb801e070 con 0x7fbdc81600b0 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc80ad6f0 con 0x7fbdc81600b0 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdb6ffd700 1 -- 172.21.15.89:0/674249657 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80038c0 con 0x7fbdc81600b0 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.807+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc80c54b0 con 0x7fbdc81600b0 2023-07-06T02:23:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.808+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc8074270 con 0x7fbdc81600b0 2023-07-06T02:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.808+0000 7fbdb6ffd700 1 -- 172.21.15.89:0/674249657 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdb8003a20 con 0x7fbdc81600b0 2023-07-06T02:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.809+0000 7fbdb6ffd700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbda806ace0 0x7fbda806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.809+0000 7fbdb6ffd700 1 -- 172.21.15.89:0/674249657 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbdb8020070 con 0x7fbdc81600b0 2023-07-06T02:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.809+0000 7fbdc659c700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbda806ace0 0x7fbda806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:37.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.809+0000 7fbdc659c700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbda806ace0 0x7fbda806d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc006fd0 tx=0x7fbdbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:37.813+0000 7fbdb6ffd700 1 -- 172.21.15.89:0/674249657 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbdb8055b00 con 0x7fbdc81600b0 2023-07-06T02:23:38.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.001+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdc80b5990 con 0x7fbdc81600b0 2023-07-06T02:23:38.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.002+0000 7fbdb6ffd700 1 -- 172.21.15.89:0/674249657 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7fbdb8059120 con 0x7fbdc81600b0 2023-07-06T02:23:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:38.005 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.004+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbda806ace0 msgr2=0x7fbda806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.004+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbda806ace0 0x7fbda806d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc006fd0 tx=0x7fbdbc008040 comp rx=0 tx=0).stop 2023-07-06T02:23:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.004+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc81600b0 msgr2=0x7fbdc80c4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.004+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc81600b0 0x7fbdc80c4f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8000c00 tx=0x7fbdb8004750 comp rx=0 tx=0).stop 2023-07-06T02:23:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 shutdown_connections 2023-07-06T02:23:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc80acc30 0x7fbdc815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8156ab0 0x7fbdc8155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbda806ace0 0x7fbda806d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 --2- 172.21.15.89:0/674249657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdc81600b0 0x7fbdc80c4f70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 >> 172.21.15.89:0/674249657 conn(0x7fbdc80cc490 msgr2=0x7fbdc80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 shutdown_connections 2023-07-06T02:23:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.005+0000 7fbdcc88f700 1 -- 172.21.15.89:0/674249657 wait complete. 2023-07-06T02:23:38.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:38.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.140+0000 7f3ebea79700 1 Processor -- start 2023-07-06T02:23:38.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.140+0000 7f3ebea79700 1 -- start start 2023-07-06T02:23:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebea79700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80705a0 0x7f3eb80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebea79700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3eb80cd180 0x7f3eb815a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebea79700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eb806dc10 0x7f3eb8157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebea79700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb806e110 con 0x7f3eb806dc10 2023-07-06T02:23:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebea79700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb806e3f0 con 0x7f3eb80cd180 2023-07-06T02:23:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebea79700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb806e6d0 con 0x7f3eb80705a0 2023-07-06T02:23:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eb806dc10 0x7f3eb8157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eb806dc10 0x7f3eb8157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33520/0 (socket says 172.21.15.89:33520) 2023-07-06T02:23:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 -- 172.21.15.89:0/2400825640 learned_addr learned my addr 172.21.15.89:0/2400825640 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3eb7fff700 1 --2- 172.21.15.89:0/2400825640 >> v2:172.21.15.172:3300/0 conn(0x7f3eb80cd180 0x7f3eb815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebc815700 1 --2- 172.21.15.89:0/2400825640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80705a0 0x7f3eb80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 -- 172.21.15.89:0/2400825640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80705a0 msgr2=0x7f3eb80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 --2- 172.21.15.89:0/2400825640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80705a0 0x7f3eb80ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 -- 172.21.15.89:0/2400825640 >> v2:172.21.15.172:3300/0 conn(0x7f3eb80cd180 msgr2=0x7f3eb815a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 --2- 172.21.15.89:0/2400825640 >> v2:172.21.15.172:3300/0 conn(0x7f3eb80cd180 0x7f3eb815a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 -- 172.21.15.89:0/2400825640 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb806e9b0 con 0x7f3eb806dc10 2023-07-06T02:23:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebc815700 1 --2- 172.21.15.89:0/2400825640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80705a0 0x7f3eb80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3ebd016700 1 --2- 172.21.15.89:0/2400825640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb8157950 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8072290 tx=0x7f3eb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aec8f0635b9762 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.141+0000 7f3eb77fe700 1 -- 172.21.15.89:0/2400825640 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000e070 con 0x7f3eb806dc10 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3eb77fe700 1 -- 172.21.15.89:0/2400825640 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb000b9c0 con 0x7f3eb806dc10 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3eb77fe700 1 -- 172.21.15.89:0/2400825640 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000bd70 con 0x7f3eb806dc10 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 -- 172.21.15.89:0/2400825640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 msgr2=0x7f3eb8157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 --2- 172.21.15.89:0/2400825640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb8157950 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8072290 tx=0x7f3eb000b330 comp rx=0 tx=0).stop 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 -- 172.21.15.89:0/2400825640 shutdown_connections 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 --2- 172.21.15.89:0/2400825640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80705a0 0x7f3eb80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 --2- 172.21.15.89:0/2400825640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb8157950 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 --2- 172.21.15.89:0/2400825640 >> v2:172.21.15.172:3300/0 conn(0x7f3eb80cd180 0x7f3eb815a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 -- 172.21.15.89:0/2400825640 >> 172.21.15.89:0/2400825640 conn(0x7f3eb80d0b00 msgr2=0x7f3eb80d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:38.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 -- 172.21.15.89:0/2400825640 shutdown_connections 2023-07-06T02:23:38.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.142+0000 7f3ebea79700 1 -- 172.21.15.89:0/2400825640 wait complete. 2023-07-06T02:23:38.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.144+0000 7f3ebea79700 1 Processor -- start 2023-07-06T02:23:38.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.144+0000 7f3ebea79700 1 -- start start 2023-07-06T02:23:38.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebea79700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb81f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebea79700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb80705a0 0x7f3eb81f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebea79700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80cd180 0x7f3eb8146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebea79700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb815c700 con 0x7f3eb806dc10 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebea79700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb815c9e0 con 0x7f3eb80705a0 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebea79700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb815ccc0 con 0x7f3eb80cd180 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb81f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb81f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33522/0 (socket says 172.21.15.89:33522) 2023-07-06T02:23:38.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 -- 172.21.15.89:0/3154578781 learned_addr learned my addr 172.21.15.89:0/3154578781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebd016700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80cd180 0x7f3eb8146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3eb7fff700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb80705a0 0x7f3eb81f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 -- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80cd180 msgr2=0x7f3eb8146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80cd180 0x7f3eb8146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 -- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb80705a0 msgr2=0x7f3eb81f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb80705a0 0x7f3eb81f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 -- 172.21.15.89:0/3154578781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb0009b30 con 0x7f3eb806dc10 2023-07-06T02:23:38.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebd016700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80cd180 0x7f3eb8146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:38.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3ebc815700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb81f3300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c740 tx=0x7f3eac00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3eb57fa700 1 -- 172.21.15.89:0/3154578781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac011800 con 0x7f3eb806dc10 2023-07-06T02:23:38.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.145+0000 7f3eb57fa700 1 -- 172.21.15.89:0/3154578781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac011960 con 0x7f3eb806dc10 2023-07-06T02:23:38.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.146+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb815cfa0 con 0x7f3eb806dc10 2023-07-06T02:23:38.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.146+0000 7f3eb57fa700 1 -- 172.21.15.89:0/3154578781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac011cd0 con 0x7f3eb806dc10 2023-07-06T02:23:38.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.146+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb81efa50 con 0x7f3eb806dc10 2023-07-06T02:23:38.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.146+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb8074270 con 0x7f3eb806dc10 2023-07-06T02:23:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.147+0000 7f3eb57fa700 1 -- 172.21.15.89:0/3154578781 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3eac021350 con 0x7f3eb806dc10 2023-07-06T02:23:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.147+0000 7f3eb57fa700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9c06ace0 0x7f3e9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.147+0000 7f3eb57fa700 1 -- 172.21.15.89:0/3154578781 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3eac089740 con 0x7f3eb806dc10 2023-07-06T02:23:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.147+0000 7f3eb7fff700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9c06ace0 0x7f3e9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.148+0000 7f3eb7fff700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9c06ace0 0x7f3e9c06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80b3100 tx=0x7f3ea4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.152+0000 7f3eb57fa700 1 -- 172.21.15.89:0/3154578781 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3eac057fa0 con 0x7f3eb806dc10 2023-07-06T02:23:38.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.339+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3eb815c840 con 0x7f3eb806dc10 2023-07-06T02:23:38.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.340+0000 7f3eb57fa700 1 -- 172.21.15.89:0/3154578781 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3eb815c840 con 0x7f3eb806dc10 2023-07-06T02:23:38.342 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:38.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9c06ace0 msgr2=0x7f3e9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9c06ace0 0x7f3e9c06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80b3100 tx=0x7f3ea4008040 comp rx=0 tx=0).stop 2023-07-06T02:23:38.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 msgr2=0x7f3eb81f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb81f3300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c740 tx=0x7f3eac00cb00 comp rx=0 tx=0).stop 2023-07-06T02:23:38.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 shutdown_connections 2023-07-06T02:23:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb80cd180 0x7f3eb8146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9c06ace0 0x7f3e9c06d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eb806dc10 0x7f3eb81f3300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 --2- 172.21.15.89:0/3154578781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb80705a0 0x7f3eb81f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 >> 172.21.15.89:0/3154578781 conn(0x7f3eb80d0b00 msgr2=0x7f3eb80d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 shutdown_connections 2023-07-06T02:23:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.342+0000 7f3ebea79700 1 -- 172.21.15.89:0/3154578781 wait complete. 2023-07-06T02:23:38.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:38.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.479+0000 7f7fba191700 1 Processor -- start 2023-07-06T02:23:38.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.480+0000 7f7fba191700 1 -- start start 2023-07-06T02:23:38.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.480+0000 7f7fba191700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 0x7f7fb40b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.480+0000 7f7fba191700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb40cab50 con 0x7f7fb415c690 2023-07-06T02:23:38.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.480+0000 7f7fba191700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb40b1e20 con 0x7f7fb406bfa0 2023-07-06T02:23:38.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.480+0000 7f7fba191700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb415de50 con 0x7f7fb40b1490 2023-07-06T02:23:38.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.480+0000 7f7fb37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7fb415c690 0x7f7fb40cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58432/0 (socket says 172.21.15.89:58432) 2023-07-06T02:23:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.480+0000 7f7fb37fe700 1 -- 172.21.15.89:0/1524539699 learned_addr learned my addr 172.21.15.89:0/1524539699 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb2ffd700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 0x7f7fb40b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357972957 0 0) 0x7f7fb40cab50 con 0x7f7fb415c690 2023-07-06T02:23:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f9c0035b0 con 0x7f7fb415c690 2023-07-06T02:23:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1747703133 0 0) 0x7f7f9c0035b0 con 0x7f7fb415c690 2023-07-06T02:23:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 msgr2=0x7f7fb40b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 0x7f7fb40b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 >> v1:172.21.15.172:6789/0 conn(0x7f7fb406bfa0 legacy=0x7f7fb406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb415db70 con 0x7f7fb415c690 2023-07-06T02:23:38.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f7fa8001670 con 0x7f7fb415c690 2023-07-06T02:23:38.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 >> v1:172.21.15.89:6789/0 conn(0x7f7fb415c690 legacy=0x7f7fb40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f9c003a30 0x7f7f9c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 0x7f7f9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f9c004d90 0x7f7f9c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb415de50 con 0x7f7f9c003a30 2023-07-06T02:23:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c00bac0 con 0x7f7f9c004d90 2023-07-06T02:23:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c00bc50 con 0x7f7fb40b1490 2023-07-06T02:23:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f9c003a30 0x7f7f9c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb2ffd700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 0x7f7f9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb3fff700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f9c004d90 0x7f7f9c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb37fe700 1 -- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 msgr2=0x7f7f9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 0x7f7f9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb37fe700 1 -- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f9c004d90 msgr2=0x7f7f9c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f9c004d90 0x7f7f9c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb37fe700 1 -- 172.21.15.89:0/1524539699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0035b0 con 0x7f7f9c003a30 2023-07-06T02:23:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb2ffd700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fb40b1490 0x7f7f9c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.481+0000 7f7fb3fff700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f9c004d90 0x7f7f9c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:38.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f9c003a30 0x7f7f9c003e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c240 tx=0x7f7f9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb48097effb28f99 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8007c10 con 0x7f7f9c003a30 2023-07-06T02:23:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa800ce00 con 0x7f7f9c003a30 2023-07-06T02:23:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fb27fc700 1 -- 172.21.15.89:0/1524539699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8005ba0 con 0x7f7f9c003a30 2023-07-06T02:23:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fba191700 1 -- 172.21.15.89:0/1524539699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f9c003a30 msgr2=0x7f7f9c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fba191700 1 --2- 172.21.15.89:0/1524539699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f9c003a30 0x7f7f9c003e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c240 tx=0x7f7f9c005880 comp rx=0 tx=0).stop 2023-07-06T02:23:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fb3fff700 1 -- 172.21.15.89:0/1524539699 reap_dead start 2023-07-06T02:23:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fba191700 1 -- 172.21.15.89:0/1524539699 shutdown_connections 2023-07-06T02:23:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fba191700 1 -- 172.21.15.89:0/1524539699 >> 172.21.15.89:0/1524539699 conn(0x7f7fb40cc490 msgr2=0x7f7fb40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fba191700 1 -- 172.21.15.89:0/1524539699 shutdown_connections 2023-07-06T02:23:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.482+0000 7f7fba191700 1 -- 172.21.15.89:0/1524539699 wait complete. 2023-07-06T02:23:38.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.483+0000 7f7fba191700 1 Processor -- start 2023-07-06T02:23:38.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fba191700 1 -- start start 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fba191700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f9c003a30 0x7f7fb40b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fba191700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb40c5300 0x7f7fb41e5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fba191700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fb41efee0 0x7f7fb41f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fba191700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40c45d0 con 0x7f7fb40c5300 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fba191700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40b2860 con 0x7f7fb41efee0 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fba191700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40b2b40 con 0x7f7f9c003a30 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fb2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb40c5300 0x7f7fb41e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fb2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb40c5300 0x7f7fb41e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33544/0 (socket says 172.21.15.89:33544) 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.484+0000 7f7fb2ffd700 1 -- 172.21.15.89:0/3616335333 learned_addr learned my addr 172.21.15.89:0/3616335333 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb3fff700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fb41efee0 0x7f7fb41f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f9c003a30 0x7f7fb40b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb2ffd700 1 -- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f9c003a30 msgr2=0x7f7fb40b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb2ffd700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f9c003a30 0x7f7fb40b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb2ffd700 1 -- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fb41efee0 msgr2=0x7f7fb41f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb2ffd700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fb41efee0 0x7f7fb41f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb2ffd700 1 -- 172.21.15.89:0/3616335333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40c4820 con 0x7f7fb40c5300 2023-07-06T02:23:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f9c003a30 0x7f7fb40b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fb2ffd700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb40c5300 0x7f7fb41e5120 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40118d0 tx=0x7f7fa4011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7f9bfff700 1 -- 172.21.15.89:0/3616335333 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4004220 con 0x7f7fb40c5300 2023-07-06T02:23:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7f9bfff700 1 -- 172.21.15.89:0/3616335333 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa4004380 con 0x7f7fb40c5300 2023-07-06T02:23:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb40c4b00 con 0x7f7fb40c5300 2023-07-06T02:23:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7f9bfff700 1 -- 172.21.15.89:0/3616335333 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40036a0 con 0x7f7fb40c5300 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.485+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb40c5770 con 0x7f7fb40c5300 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.486+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb4074270 con 0x7f7fb40c5300 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.487+0000 7f7f9bfff700 1 -- 172.21.15.89:0/3616335333 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fa4003820 con 0x7f7fb40c5300 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.487+0000 7f7f9bfff700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7fa006ace0 0x7f7fa006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.487+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7fa006ace0 0x7f7fa006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.487+0000 7f7f9bfff700 1 -- 172.21.15.89:0/3616335333 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7fa4089ab0 con 0x7f7fb40c5300 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.488+0000 7f7fb37fe700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7fa006ace0 0x7f7fa006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8001930 tx=0x7f7fa8007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.492+0000 7f7f9bfff700 1 -- 172.21.15.89:0/3616335333 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7fa4054870 con 0x7f7fb40c5300 2023-07-06T02:23:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.680+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fb406bd80 con 0x7f7fb40c5300 2023-07-06T02:23:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.680+0000 7f7f9bfff700 1 -- 172.21.15.89:0/3616335333 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fa4057e90 con 0x7f7fb40c5300 2023-07-06T02:23:38.682 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7fa006ace0 msgr2=0x7f7fa006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7fa006ace0 0x7f7fa006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8001930 tx=0x7f7fa8007000 comp rx=0 tx=0).stop 2023-07-06T02:23:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb40c5300 msgr2=0x7f7fb41e5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb40c5300 0x7f7fb41e5120 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40118d0 tx=0x7f7fa4011be0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 shutdown_connections 2023-07-06T02:23:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fb41efee0 0x7f7fb41f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f9c003a30 0x7f7fb40b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7fa006ace0 0x7f7fa006d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 --2- 172.21.15.89:0/3616335333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb40c5300 0x7f7fb41e5120 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 >> 172.21.15.89:0/3616335333 conn(0x7f7fb40cc490 msgr2=0x7f7fb40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:38.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 shutdown_connections 2023-07-06T02:23:38.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.682+0000 7f7fba191700 1 -- 172.21.15.89:0/3616335333 wait complete. 2023-07-06T02:23:38.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:38.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.816+0000 7f4cc4bc2700 1 Processor -- start 2023-07-06T02:23:38.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.816+0000 7f4cc4bc2700 1 -- start start 2023-07-06T02:23:38.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.817+0000 7f4cc4bc2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cc00705a0 0x7f4cc00ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.817+0000 7f4cc4bc2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4cc006dc10 0x7f4cc0157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.817+0000 7f4cc4bc2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc006e110 con 0x7f4cc00cd180 2023-07-06T02:23:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.817+0000 7f4cc4bc2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc006e3f0 con 0x7f4cc006dc10 2023-07-06T02:23:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.817+0000 7f4cc4bc2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc006e6d0 con 0x7f4cc00705a0 2023-07-06T02:23:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbdd9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4cc00cd180 0x7f4cc015a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58452/0 (socket says 172.21.15.89:58452) 2023-07-06T02:23:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbdd9b700 1 -- 172.21.15.89:0/1183996109 learned_addr learned my addr 172.21.15.89:0/1183996109 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059357655 0 0) 0x7f4cc006e110 con 0x7f4cc00cd180 2023-07-06T02:23:38.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbe59c700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cc00705a0 0x7f4cc00ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca80035b0 con 0x7f4cc00cd180 2023-07-06T02:23:38.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbed9d700 1 --2- 172.21.15.89:0/1183996109 >> v2:172.21.15.172:3300/0 conn(0x7f4cc006dc10 0x7f4cc0157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3649257728 0 0) 0x7f4ca80035b0 con 0x7f4cc00cd180 2023-07-06T02:23:38.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cc00705a0 msgr2=0x7f4cc00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cc00705a0 0x7f4cc00ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 >> v2:172.21.15.172:3300/0 conn(0x7f4cc006dc10 msgr2=0x7f4cc0157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 --2- 172.21.15.89:0/1183996109 >> v2:172.21.15.172:3300/0 conn(0x7f4cc006dc10 0x7f4cc0157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc006e9b0 con 0x7f4cc00cd180 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f4cb0001670 con 0x7f4cc00cd180 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 >> v1:172.21.15.89:6789/0 conn(0x7f4cc00cd180 legacy=0x7f4cc015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ca8003a00 0x7f4ca8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004430 0x7f4ca80048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8004fc0 0x7f4ca800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc006e3f0 con 0x7f4ca8004fc0 2023-07-06T02:23:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc006e6d0 con 0x7f4ca8003a00 2023-07-06T02:23:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca800bf50 con 0x7f4ca8004430 2023-07-06T02:23:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbe59c700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cc00705a0 0x7f4cc00ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbed9d700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8004fc0 0x7f4ca800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbdd9b700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004430 0x7f4ca80048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.818+0000 7f4cbe59c700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ca8003a00 0x7f4ca8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbed9d700 1 -- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004430 msgr2=0x7f4ca80048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbed9d700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004430 0x7f4ca80048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbed9d700 1 -- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ca8003a00 msgr2=0x7f4ca8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbed9d700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ca8003a00 0x7f4ca8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbed9d700 1 -- 172.21.15.89:0/1183996109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80035b0 con 0x7f4ca8004fc0 2023-07-06T02:23:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbed9d700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8004fc0 0x7f4ca800ba10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800e170 tx=0x7f4ca8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7773018861090dba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb800ec90 con 0x7f4ca8004fc0 2023-07-06T02:23:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb800f040 con 0x7f4ca8004fc0 2023-07-06T02:23:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbd59a700 1 -- 172.21.15.89:0/1183996109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8014480 con 0x7f4ca8004fc0 2023-07-06T02:23:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1183996109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8004fc0 msgr2=0x7f4ca800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cc4bc2700 1 --2- 172.21.15.89:0/1183996109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ca8004fc0 0x7f4ca800ba10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800e170 tx=0x7f4ca8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cbed9d700 1 -- 172.21.15.89:0/1183996109 reap_dead start 2023-07-06T02:23:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1183996109 shutdown_connections 2023-07-06T02:23:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1183996109 >> 172.21.15.89:0/1183996109 conn(0x7f4cc00d0b00 msgr2=0x7f4cc00d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1183996109 shutdown_connections 2023-07-06T02:23:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.819+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1183996109 wait complete. 2023-07-06T02:23:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 Processor -- start 2023-07-06T02:23:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 -- start start 2023-07-06T02:23:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004fc0 0x7f4cc00abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cc0150dc0 0x7f4cc00b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cc00b5780 0x7f4cc00c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00b5c20 con 0x7f4cc0150dc0 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00c8c70 con 0x7f4cc00b5780 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.821+0000 7f4cc4bc2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00ac160 con 0x7f4ca8004fc0 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cc0150dc0 0x7f4cc00b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cc0150dc0 0x7f4cc00b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33564/0 (socket says 172.21.15.89:33564) 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 -- 172.21.15.89:0/1375643827 learned_addr learned my addr 172.21.15.89:0/1375643827 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbe59c700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004fc0 0x7f4cc00abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbed9d700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cc00b5780 0x7f4cc00c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 -- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004fc0 msgr2=0x7f4cc00abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004fc0 0x7f4cc00abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 -- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cc00b5780 msgr2=0x7f4cc00c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cc00b5780 0x7f4cc00c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 -- 172.21.15.89:0/1375643827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc006bb30 con 0x7f4cc0150dc0 2023-07-06T02:23:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cbdd9b700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cc0150dc0 0x7f4cc00b5240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0007310 tx=0x7f4cb0007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4caeffd700 1 -- 172.21.15.89:0/1375643827 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000f5e0 con 0x7f4cc0150dc0 2023-07-06T02:23:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4caeffd700 1 -- 172.21.15.89:0/1375643827 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb000f740 con 0x7f4cc0150dc0 2023-07-06T02:23:38.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc00c8990 con 0x7f4cc0150dc0 2023-07-06T02:23:38.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4caeffd700 1 -- 172.21.15.89:0/1375643827 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000faf0 con 0x7f4cc0150dc0 2023-07-06T02:23:38.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.822+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc006c0b0 con 0x7f4cc0150dc0 2023-07-06T02:23:38.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.823+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc0074270 con 0x7f4cc0150dc0 2023-07-06T02:23:38.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.824+0000 7f4caeffd700 1 -- 172.21.15.89:0/1375643827 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4cb000fc50 con 0x7f4cc0150dc0 2023-07-06T02:23:38.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.824+0000 7f4caeffd700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ca006ace0 0x7f4ca006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:38.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.824+0000 7f4caeffd700 1 -- 172.21.15.89:0/1375643827 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4cb0089690 con 0x7f4cc0150dc0 2023-07-06T02:23:38.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.824+0000 7f4cbe59c700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ca006ace0 0x7f4ca006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:38.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.824+0000 7f4cbe59c700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ca006ace0 0x7f4ca006d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006f10 tx=0x7f4cb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:38.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:38.828+0000 7f4caeffd700 1 -- 172.21.15.89:0/1375643827 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4cb00544e0 con 0x7f4cc0150dc0 2023-07-06T02:23:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.015+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4cc006f590 con 0x7f4cc0150dc0 2023-07-06T02:23:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.015+0000 7f4caeffd700 1 -- 172.21.15.89:0/1375643827 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4cb0057b00 con 0x7f4cc0150dc0 2023-07-06T02:23:39.017 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ca006ace0 msgr2=0x7f4ca006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ca006ace0 0x7f4ca006d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006f10 tx=0x7f4cb4008040 comp rx=0 tx=0).stop 2023-07-06T02:23:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cc0150dc0 msgr2=0x7f4cc00b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cc0150dc0 0x7f4cc00b5240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0007310 tx=0x7f4cb0007620 comp rx=0 tx=0).stop 2023-07-06T02:23:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 shutdown_connections 2023-07-06T02:23:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cc00b5780 0x7f4cc00c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ca8004fc0 0x7f4cc00abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4ca006ace0 0x7f4ca006d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 --2- 172.21.15.89:0/1375643827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cc0150dc0 0x7f4cc00b5240 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 >> 172.21.15.89:0/1375643827 conn(0x7f4cc00d0b00 msgr2=0x7f4cc00cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:39.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 shutdown_connections 2023-07-06T02:23:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.017+0000 7f4cc4bc2700 1 -- 172.21.15.89:0/1375643827 wait complete. 2023-07-06T02:23:39.031 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:39.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.151+0000 7f98b4a59700 1 Processor -- start 2023-07-06T02:23:39.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.151+0000 7f98b4a59700 1 -- start start 2023-07-06T02:23:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98b4a59700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b0070f20 0x7f98b00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98b4a59700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f98b00712f0 0x7f98b006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98b4a59700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b00c9fa0 con 0x7f98b006e790 2023-07-06T02:23:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98b4a59700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b006f290 con 0x7f98b00712f0 2023-07-06T02:23:39.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98b4a59700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b006ecd0 con 0x7f98b0070f20 2023-07-06T02:23:39.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f98b00712f0 0x7f98b006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f98b00712f0 0x7f98b006e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33574/0 (socket says 172.21.15.89:33574) 2023-07-06T02:23:39.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 -- 172.21.15.89:0/2200559866 learned_addr learned my addr 172.21.15.89:0/2200559866 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:39.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98ae59c700 1 --2- 172.21.15.89:0/2200559866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b0070f20 0x7f98b00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 -- 172.21.15.89:0/2200559866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b0070f20 msgr2=0x7f98b00c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:39.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 --2- 172.21.15.89:0/2200559866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b0070f20 0x7f98b00c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 -- 172.21.15.89:0/2200559866 >> v1:172.21.15.172:6789/0 conn(0x7f98b006e790 legacy=0x7f98b006b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:39.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 -- 172.21.15.89:0/2200559866 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b006efb0 con 0x7f98b00712f0 2023-07-06T02:23:39.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.152+0000 7f98add9b700 1 --2- 172.21.15.89:0/2200559866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b00712f0 0x7f98b006e250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098f0 tx=0x7f98a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9c4e3c5f3b6b51 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98ad59a700 1 -- 172.21.15.89:0/2200559866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0004230 con 0x7f98b00712f0 2023-07-06T02:23:39.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98ad59a700 1 -- 172.21.15.89:0/2200559866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0004390 con 0x7f98b00712f0 2023-07-06T02:23:39.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98ad59a700 1 -- 172.21.15.89:0/2200559866 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a00036a0 con 0x7f98b00712f0 2023-07-06T02:23:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 -- 172.21.15.89:0/2200559866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b00712f0 msgr2=0x7f98b006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2200559866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b00712f0 0x7f98b006e250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098f0 tx=0x7f98a0009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 -- 172.21.15.89:0/2200559866 shutdown_connections 2023-07-06T02:23:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2200559866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b0070f20 0x7f98b00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2200559866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b00712f0 0x7f98b006e250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 -- 172.21.15.89:0/2200559866 >> 172.21.15.89:0/2200559866 conn(0x7f98b00cf1c0 msgr2=0x7f98b00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 -- 172.21.15.89:0/2200559866 shutdown_connections 2023-07-06T02:23:39.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.153+0000 7f98b4a59700 1 -- 172.21.15.89:0/2200559866 wait complete. 2023-07-06T02:23:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 Processor -- start 2023-07-06T02:23:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 -- start start 2023-07-06T02:23:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b006e790 0x7f98b00b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b0070f20 0x7f98b01f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b00712f0 0x7f98b01f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b01492e0 con 0x7f98b006e790 2023-07-06T02:23:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b014cb80 con 0x7f98b0070f20 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98b4a59700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00b1530 con 0x7f98b00712f0 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98ae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b006e790 0x7f98b00b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98ae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b006e790 0x7f98b00b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33576/0 (socket says 172.21.15.89:33576) 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.155+0000 7f98ae59c700 1 -- 172.21.15.89:0/2104350657 learned_addr learned my addr 172.21.15.89:0/2104350657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98ae59c700 1 -- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b00712f0 msgr2=0x7f98b01f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98ae59c700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b00712f0 0x7f98b01f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98ae59c700 1 -- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b0070f20 msgr2=0x7f98b01f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98add9b700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b0070f20 0x7f98b01f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98ae59c700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b0070f20 0x7f98b01f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98ae59c700 1 -- 172.21.15.89:0/2104350657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a0009550 con 0x7f98b006e790 2023-07-06T02:23:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98add9b700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b0070f20 0x7f98b01f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98ae59c700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b006e790 0x7f98b00b0ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e840 tx=0x7f98a400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f989effd700 1 -- 172.21.15.89:0/2104350657 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4013610 con 0x7f98b006e790 2023-07-06T02:23:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f989effd700 1 -- 172.21.15.89:0/2104350657 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a4013770 con 0x7f98b006e790 2023-07-06T02:23:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b00c83c0 con 0x7f98b006e790 2023-07-06T02:23:39.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f989effd700 1 -- 172.21.15.89:0/2104350657 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4013b20 con 0x7f98b006e790 2023-07-06T02:23:39.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.156+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b00c8980 con 0x7f98b006e790 2023-07-06T02:23:39.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.157+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b0074270 con 0x7f98b006e790 2023-07-06T02:23:39.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.157+0000 7f989effd700 1 -- 172.21.15.89:0/2104350657 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98a4013c80 con 0x7f98b006e790 2023-07-06T02:23:39.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.158+0000 7f989effd700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f989006ace0 0x7f989006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.158+0000 7f989effd700 1 -- 172.21.15.89:0/2104350657 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f98a4089440 con 0x7f98b006e790 2023-07-06T02:23:39.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.158+0000 7f98add9b700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f989006ace0 0x7f989006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.158+0000 7f98add9b700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f989006ace0 0x7f989006d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98b00c9100 tx=0x7f98a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.162+0000 7f989effd700 1 -- 172.21.15.89:0/2104350657 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f98a4054290 con 0x7f98b006e790 2023-07-06T02:23:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.349+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98b00ae900 con 0x7f98b006e790 2023-07-06T02:23:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.349+0000 7f989effd700 1 -- 172.21.15.89:0/2104350657 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98a40578b0 con 0x7f98b006e790 2023-07-06T02:23:39.352 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:39.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f989006ace0 msgr2=0x7f989006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f989006ace0 0x7f989006d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98b00c9100 tx=0x7f98a0008030 comp rx=0 tx=0).stop 2023-07-06T02:23:39.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b006e790 msgr2=0x7f98b00b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b006e790 0x7f98b00b0ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e840 tx=0x7f98a400ec00 comp rx=0 tx=0).stop 2023-07-06T02:23:39.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 shutdown_connections 2023-07-06T02:23:39.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b00712f0 0x7f98b01f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f989006ace0 0x7f989006d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98b006e790 0x7f98b00b0ff0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 --2- 172.21.15.89:0/2104350657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b0070f20 0x7f98b01f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 >> 172.21.15.89:0/2104350657 conn(0x7f98b00cf1c0 msgr2=0x7f98b0155770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:39.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 shutdown_connections 2023-07-06T02:23:39.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.352+0000 7f98b4a59700 1 -- 172.21.15.89:0/2104350657 wait complete. 2023-07-06T02:23:39.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:39.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.487+0000 7fdfcba73700 1 Processor -- start 2023-07-06T02:23:39.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.487+0000 7fdfcba73700 1 -- start start 2023-07-06T02:23:39.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfcba73700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdfc40b1490 0x7fdfc40b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfcba73700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdfc406bfa0 0x7fdfc406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfcba73700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc40cab50 con 0x7fdfc415c690 2023-07-06T02:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfcba73700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc415de50 con 0x7fdfc406bfa0 2023-07-06T02:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfcba73700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc40b1e50 con 0x7fdfc40b1490 2023-07-06T02:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfca010700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdfc406bfa0 0x7fdfc406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfc980f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdfc415c690 0x7fdfc40cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58486/0 (socket says 172.21.15.89:58486) 2023-07-06T02:23:39.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfc980f700 1 -- 172.21.15.89:0/869822774 learned_addr learned my addr 172.21.15.89:0/869822774 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfc900e700 1 --2- 172.21.15.89:0/869822774 >> v2:172.21.15.172:3300/0 conn(0x7fdfc40b1490 0x7fdfc40b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfca010700 1 -- 172.21.15.89:0/869822774 >> v2:172.21.15.172:3300/0 conn(0x7fdfc40b1490 msgr2=0x7fdfc40b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfca010700 1 --2- 172.21.15.89:0/869822774 >> v2:172.21.15.172:3300/0 conn(0x7fdfc40b1490 0x7fdfc40b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfca010700 1 -- 172.21.15.89:0/869822774 >> v1:172.21.15.89:6789/0 conn(0x7fdfc415c690 legacy=0x7fdfc40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfca010700 1 -- 172.21.15.89:0/869822774 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc406c8e0 con 0x7fdfc406bfa0 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.488+0000 7fdfc880d700 1 -- 172.21.15.89:0/869822774 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511750451 0 0) 0x7fdfc40cab50 con 0x7fdfc415c690 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfca010700 1 --2- 172.21.15.89:0/869822774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc406c3a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4070180 tx=0x7fdfbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9b70249954cd80 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfc880d700 1 -- 172.21.15.89:0/869822774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00e070 con 0x7fdfc406bfa0 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfc880d700 1 -- 172.21.15.89:0/869822774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc00bbd0 con 0x7fdfc406bfa0 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfc880d700 1 -- 172.21.15.89:0/869822774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc012430 con 0x7fdfc406bfa0 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 -- 172.21.15.89:0/869822774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 msgr2=0x7fdfc406c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 --2- 172.21.15.89:0/869822774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc406c3a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4070180 tx=0x7fdfbc00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 -- 172.21.15.89:0/869822774 shutdown_connections 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 --2- 172.21.15.89:0/869822774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc406c3a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 --2- 172.21.15.89:0/869822774 >> v2:172.21.15.172:3300/0 conn(0x7fdfc40b1490 0x7fdfc40b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 -- 172.21.15.89:0/869822774 >> 172.21.15.89:0/869822774 conn(0x7fdfc40cc490 msgr2=0x7fdfc40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 -- 172.21.15.89:0/869822774 shutdown_connections 2023-07-06T02:23:39.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.489+0000 7fdfcba73700 1 -- 172.21.15.89:0/869822774 wait complete. 2023-07-06T02:23:39.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.490+0000 7fdfcba73700 1 Processor -- start 2023-07-06T02:23:39.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfcba73700 1 -- start start 2023-07-06T02:23:39.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfcba73700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc41eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfcba73700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40b1490 0x7fdfc414cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfcba73700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc415c690 0x7fdfc40c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfcba73700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40c6e70 con 0x7fdfc406bfa0 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfcba73700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40c77f0 con 0x7fdfc415c690 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfcba73700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40c7ad0 con 0x7fdfc40b1490 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfc980f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc41eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfc980f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc41eedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33590/0 (socket says 172.21.15.89:33590) 2023-07-06T02:23:39.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.491+0000 7fdfc980f700 1 -- 172.21.15.89:0/3744927806 learned_addr learned my addr 172.21.15.89:0/3744927806 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfc980f700 1 -- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40b1490 msgr2=0x7fdfc414cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfc900e700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40b1490 0x7fdfc414cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfca010700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc415c690 0x7fdfc40c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfc980f700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40b1490 0x7fdfc414cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfc980f700 1 -- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc415c690 msgr2=0x7fdfc40c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfc980f700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc415c690 0x7fdfc40c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfc980f700 1 -- 172.21.15.89:0/3744927806 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc009a70 con 0x7fdfc406bfa0 2023-07-06T02:23:39.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfc980f700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc41eedf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40011e0 tx=0x7fdfb400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfba7fc700 1 -- 172.21.15.89:0/3744927806 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4004c80 con 0x7fdfc406bfa0 2023-07-06T02:23:39.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfba7fc700 1 -- 172.21.15.89:0/3744927806 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb4004e00 con 0x7fdfc406bfa0 2023-07-06T02:23:39.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc40c7db0 con 0x7fdfc406bfa0 2023-07-06T02:23:39.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfba7fc700 1 -- 172.21.15.89:0/3744927806 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4007310 con 0x7fdfc406bfa0 2023-07-06T02:23:39.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.492+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc40c8370 con 0x7fdfc406bfa0 2023-07-06T02:23:39.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.493+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc40b3e70 con 0x7fdfc406bfa0 2023-07-06T02:23:39.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.493+0000 7fdfba7fc700 1 -- 172.21.15.89:0/3744927806 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfb4007470 con 0x7fdfc406bfa0 2023-07-06T02:23:39.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.494+0000 7fdfba7fc700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdfa806ace0 0x7fdfa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.494+0000 7fdfba7fc700 1 -- 172.21.15.89:0/3744927806 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdfb4089510 con 0x7fdfc406bfa0 2023-07-06T02:23:39.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.494+0000 7fdfc900e700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdfa806ace0 0x7fdfa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.494+0000 7fdfc900e700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdfa806ace0 0x7fdfa806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b1b30 tx=0x7fdfc0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.498+0000 7fdfba7fc700 1 -- 172.21.15.89:0/3744927806 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdfb40542d0 con 0x7fdfc406bfa0 2023-07-06T02:23:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.686+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdfc40c8590 con 0x7fdfc406bfa0 2023-07-06T02:23:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.686+0000 7fdfba7fc700 1 -- 172.21.15.89:0/3744927806 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdfb40578f0 con 0x7fdfc406bfa0 2023-07-06T02:23:39.689 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdfa806ace0 msgr2=0x7fdfa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdfa806ace0 0x7fdfa806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b1b30 tx=0x7fdfc0008040 comp rx=0 tx=0).stop 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 msgr2=0x7fdfc41eedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc41eedf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40011e0 tx=0x7fdfb400cae0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 shutdown_connections 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc415c690 0x7fdfc40c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdfa806ace0 0x7fdfa806d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfc406bfa0 0x7fdfc41eedf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 --2- 172.21.15.89:0/3744927806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40b1490 0x7fdfc414cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 >> 172.21.15.89:0/3744927806 conn(0x7fdfc40cc490 msgr2=0x7fdfc415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 shutdown_connections 2023-07-06T02:23:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.688+0000 7fdfcba73700 1 -- 172.21.15.89:0/3744927806 wait complete. 2023-07-06T02:23:39.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:39.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.821+0000 7f42a9ea7700 1 Processor -- start 2023-07-06T02:23:39.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.821+0000 7f42a9ea7700 1 -- start start 2023-07-06T02:23:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.821+0000 7f42a9ea7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42a406dc10 0x7f42a415a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.821+0000 7f42a9ea7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42a415adb0 0x7f42a4157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.821+0000 7f42a9ea7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40cd350 con 0x7f42a40705a0 2023-07-06T02:23:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.821+0000 7f42a9ea7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a406e110 con 0x7f42a415adb0 2023-07-06T02:23:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.821+0000 7f42a9ea7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a406e3f0 con 0x7f42a406dc10 2023-07-06T02:23:39.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42a415adb0 0x7f42a4157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f42a40705a0 0x7f42a40ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58500/0 (socket says 172.21.15.89:58500) 2023-07-06T02:23:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a37fe700 1 -- 172.21.15.89:0/2727697473 learned_addr learned my addr 172.21.15.89:0/2727697473 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a3fff700 1 -- 172.21.15.89:0/2727697473 >> v2:172.21.15.172:3300/0 conn(0x7f42a406dc10 msgr2=0x7f42a415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a2ffd700 1 --2- 172.21.15.89:0/2727697473 >> v2:172.21.15.172:3300/0 conn(0x7f42a406dc10 0x7f42a415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a3fff700 1 --2- 172.21.15.89:0/2727697473 >> v2:172.21.15.172:3300/0 conn(0x7f42a406dc10 0x7f42a415a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a3fff700 1 -- 172.21.15.89:0/2727697473 >> v1:172.21.15.89:6789/0 conn(0x7f42a40705a0 legacy=0x7f42a40ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a27fc700 1 -- 172.21.15.89:0/2727697473 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124795340 0 0) 0x7f42a40cd350 con 0x7f42a40705a0 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a3fff700 1 -- 172.21.15.89:0/2727697473 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a406e6d0 con 0x7f42a415adb0 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a3fff700 1 --2- 172.21.15.89:0/2727697473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a415adb0 0x7f42a4157950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f429c009d90 tx=0x7f429c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6362177beecb33 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a27fc700 1 -- 172.21.15.89:0/2727697473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00e070 con 0x7f42a415adb0 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a27fc700 1 -- 172.21.15.89:0/2727697473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c00bb90 con 0x7f42a415adb0 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.822+0000 7f42a27fc700 1 -- 172.21.15.89:0/2727697473 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c012430 con 0x7f42a415adb0 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2727697473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a415adb0 msgr2=0x7f42a4157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2727697473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a415adb0 0x7f42a4157950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f429c009d90 tx=0x7f429c00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2727697473 shutdown_connections 2023-07-06T02:23:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2727697473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a415adb0 0x7f42a4157950 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2727697473 >> v2:172.21.15.172:3300/0 conn(0x7f42a406dc10 0x7f42a415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2727697473 >> 172.21.15.89:0/2727697473 conn(0x7f42a40d0b00 msgr2=0x7f42a40d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2727697473 shutdown_connections 2023-07-06T02:23:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.823+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2727697473 wait complete. 2023-07-06T02:23:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.824+0000 7f42a9ea7700 1 Processor -- start 2023-07-06T02:23:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.824+0000 7f42a9ea7700 1 -- start start 2023-07-06T02:23:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a9ea7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406dc10 0x7f42a414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a9ea7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a40705a0 0x7f42a40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a9ea7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a415adb0 0x7f42a40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a9ea7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a41567b0 con 0x7f42a40705a0 2023-07-06T02:23:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a9ea7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c95e0 con 0x7f42a415adb0 2023-07-06T02:23:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a9ea7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a414ea90 con 0x7f42a406dc10 2023-07-06T02:23:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a40705a0 0x7f42a40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a40705a0 0x7f42a40c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33604/0 (socket says 172.21.15.89:33604) 2023-07-06T02:23:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 -- 172.21.15.89:0/2916859258 learned_addr learned my addr 172.21.15.89:0/2916859258 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a3fff700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a415adb0 0x7f42a40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a37fe700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406dc10 0x7f42a414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 -- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406dc10 msgr2=0x7f42a414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406dc10 0x7f42a414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 -- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a415adb0 msgr2=0x7f42a40ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a415adb0 0x7f42a40ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 -- 172.21.15.89:0/2916859258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c009a70 con 0x7f42a40705a0 2023-07-06T02:23:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.825+0000 7f42a2ffd700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a40705a0 0x7f42a40c87d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c7f0 tx=0x7f428c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.826+0000 7f429bfff700 1 -- 172.21.15.89:0/2916859258 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c011800 con 0x7f42a40705a0 2023-07-06T02:23:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.826+0000 7f429bfff700 1 -- 172.21.15.89:0/2916859258 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c011960 con 0x7f42a40705a0 2023-07-06T02:23:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.826+0000 7f429bfff700 1 -- 172.21.15.89:0/2916859258 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c011cf0 con 0x7f42a40705a0 2023-07-06T02:23:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.826+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a40c9020 con 0x7f42a40705a0 2023-07-06T02:23:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.826+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a41569e0 con 0x7f42a40705a0 2023-07-06T02:23:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.827+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a4074270 con 0x7f42a40705a0 2023-07-06T02:23:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.827+0000 7f429bfff700 1 -- 172.21.15.89:0/2916859258 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f428c018360 con 0x7f42a40705a0 2023-07-06T02:23:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.827+0000 7f429bfff700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f428406ace0 0x7f428406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.827+0000 7f429bfff700 1 -- 172.21.15.89:0/2916859258 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f428c0897a0 con 0x7f42a40705a0 2023-07-06T02:23:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.827+0000 7f42a37fe700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f428406ace0 0x7f428406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.828+0000 7f42a37fe700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f428406ace0 0x7f428406d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4294000f60 tx=0x7f4294007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:39.832+0000 7f429bfff700 1 -- 172.21.15.89:0/2916859258 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f428c058000 con 0x7f42a40705a0 2023-07-06T02:23:40.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.022+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42a41517a0 con 0x7f42a40705a0 2023-07-06T02:23:40.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.024+0000 7f429bfff700 1 -- 172.21.15.89:0/2916859258 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f428c057b90 con 0x7f42a40705a0 2023-07-06T02:23:40.026 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:40.026 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":24,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:22:57.836980+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:40.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f428406ace0 msgr2=0x7f428406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f428406ace0 0x7f428406d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4294000f60 tx=0x7f4294007be0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a40705a0 msgr2=0x7f42a40c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a40705a0 0x7f42a40c87d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c7f0 tx=0x7f428c00cbb0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 shutdown_connections 2023-07-06T02:23:40.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a415adb0 0x7f42a40ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406dc10 0x7f42a414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f428406ace0 0x7f428406d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 --2- 172.21.15.89:0/2916859258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42a40705a0 0x7f42a40c87d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 >> 172.21.15.89:0/2916859258 conn(0x7f42a40d0b00 msgr2=0x7f42a40b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:40.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 shutdown_connections 2023-07-06T02:23:40.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.026+0000 7f42a9ea7700 1 -- 172.21.15.89:0/2916859258 wait complete. 2023-07-06T02:23:40.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:40.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.161+0000 7f4e54025700 1 Processor -- start 2023-07-06T02:23:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.161+0000 7f4e54025700 1 -- start start 2023-07-06T02:23:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e54025700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e4c0b1490 0x7f4e4c0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e54025700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e4c06bfa0 0x7f4e4c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e54025700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0cab50 con 0x7f4e4c15c690 2023-07-06T02:23:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e54025700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c15de50 con 0x7f4e4c06bfa0 2023-07-06T02:23:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e54025700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b1e50 con 0x7f4e4c0b1490 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e525c2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e4c06bfa0 0x7f4e4c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e51dc1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4e4c15c690 0x7f4e4c0cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58518/0 (socket says 172.21.15.89:58518) 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.162+0000 7f4e51dc1700 1 -- 172.21.15.89:0/2085841462 learned_addr learned my addr 172.21.15.89:0/2085841462 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e525c2700 1 -- 172.21.15.89:0/2085841462 >> v2:172.21.15.172:3300/0 conn(0x7f4e4c0b1490 msgr2=0x7f4e4c0b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e525c2700 1 --2- 172.21.15.89:0/2085841462 >> v2:172.21.15.172:3300/0 conn(0x7f4e4c0b1490 0x7f4e4c0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e525c2700 1 -- 172.21.15.89:0/2085841462 >> v1:172.21.15.89:6789/0 conn(0x7f4e4c15c690 legacy=0x7f4e4c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e525c2700 1 -- 172.21.15.89:0/2085841462 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c06c8e0 con 0x7f4e4c06bfa0 2023-07-06T02:23:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e50dbf700 1 -- 172.21.15.89:0/2085841462 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645848316 0 0) 0x7f4e4c0cab50 con 0x7f4e4c15c690 2023-07-06T02:23:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e525c2700 1 --2- 172.21.15.89:0/2085841462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c06bfa0 0x7f4e4c06c3a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c070180 tx=0x7f4e4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27cec68a51927b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e50dbf700 1 -- 172.21.15.89:0/2085841462 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400e070 con 0x7f4e4c06bfa0 2023-07-06T02:23:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e50dbf700 1 -- 172.21.15.89:0/2085841462 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400bbd0 con 0x7f4e4c06bfa0 2023-07-06T02:23:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e50dbf700 1 -- 172.21.15.89:0/2085841462 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44012430 con 0x7f4e4c06bfa0 2023-07-06T02:23:40.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e54025700 1 -- 172.21.15.89:0/2085841462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c06bfa0 msgr2=0x7f4e4c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.163+0000 7f4e54025700 1 --2- 172.21.15.89:0/2085841462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c06bfa0 0x7f4e4c06c3a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c070180 tx=0x7f4e4400b330 comp rx=0 tx=0).stop 2023-07-06T02:23:40.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.164+0000 7f4e54025700 1 -- 172.21.15.89:0/2085841462 shutdown_connections 2023-07-06T02:23:40.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.164+0000 7f4e54025700 1 --2- 172.21.15.89:0/2085841462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c06bfa0 0x7f4e4c06c3a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.164+0000 7f4e54025700 1 --2- 172.21.15.89:0/2085841462 >> v2:172.21.15.172:3300/0 conn(0x7f4e4c0b1490 0x7f4e4c0b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.164+0000 7f4e54025700 1 -- 172.21.15.89:0/2085841462 >> 172.21.15.89:0/2085841462 conn(0x7f4e4c0cc490 msgr2=0x7f4e4c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.164+0000 7f4e54025700 1 -- 172.21.15.89:0/2085841462 shutdown_connections 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.164+0000 7f4e54025700 1 -- 172.21.15.89:0/2085841462 wait complete. 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.165+0000 7f4e54025700 1 Processor -- start 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.165+0000 7f4e54025700 1 -- start start 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.165+0000 7f4e54025700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4c06bfa0 0x7f4e4c0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.165+0000 7f4e54025700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c0b1490 0x7f4e4c14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e54025700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4c15c690 0x7f4e4c14b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e54025700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b5160 con 0x7f4e4c0b1490 2023-07-06T02:23:40.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e54025700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c155430 con 0x7f4e4c15c690 2023-07-06T02:23:40.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e54025700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b4b10 con 0x7f4e4c06bfa0 2023-07-06T02:23:40.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c0b1490 0x7f4e4c14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c0b1490 0x7f4e4c14fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33624/0 (socket says 172.21.15.89:33624) 2023-07-06T02:23:40.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 -- 172.21.15.89:0/3025025416 learned_addr learned my addr 172.21.15.89:0/3025025416 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:40.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e525c2700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4c15c690 0x7f4e4c14b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e51dc1700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4c06bfa0 0x7f4e4c0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 -- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4c06bfa0 msgr2=0x7f4e4c0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4c06bfa0 0x7f4e4c0b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 -- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4c15c690 msgr2=0x7f4e4c14b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4c15c690 0x7f4e4c14b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 -- 172.21.15.89:0/3025025416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44009a70 con 0x7f4e4c0b1490 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e515c0700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c0b1490 0x7f4e4c14fcd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800b530 tx=0x7f4e4800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e427fc700 1 -- 172.21.15.89:0/3025025416 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48011830 con 0x7f4e4c0b1490 2023-07-06T02:23:40.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e427fc700 1 -- 172.21.15.89:0/3025025416 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48011990 con 0x7f4e4c0b1490 2023-07-06T02:23:40.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c0b4df0 con 0x7f4e4c0b1490 2023-07-06T02:23:40.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e427fc700 1 -- 172.21.15.89:0/3025025416 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48011d40 con 0x7f4e4c0b1490 2023-07-06T02:23:40.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.166+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c150210 con 0x7f4e4c0b1490 2023-07-06T02:23:40.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.167+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c0b5a20 con 0x7f4e4c0b1490 2023-07-06T02:23:40.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.168+0000 7f4e427fc700 1 -- 172.21.15.89:0/3025025416 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e4801a400 con 0x7f4e4c0b1490 2023-07-06T02:23:40.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.168+0000 7f4e427fc700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e3006ace0 0x7f4e3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.168+0000 7f4e427fc700 1 -- 172.21.15.89:0/3025025416 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4e48089760 con 0x7f4e4c0b1490 2023-07-06T02:23:40.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.168+0000 7f4e51dc1700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e3006ace0 0x7f4e3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.169+0000 7f4e51dc1700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e3006ace0 0x7f4e3006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c007fd0 tx=0x7f4e3c007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.172+0000 7f4e427fc700 1 -- 172.21.15.89:0/3025025416 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4e480545e0 con 0x7f4e4c0b1490 2023-07-06T02:23:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.358+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4e4c0b52a0 con 0x7f4e4c0b1490 2023-07-06T02:23:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.359+0000 7f4e427fc700 1 -- 172.21.15.89:0/3025025416 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4e4c0b52a0 con 0x7f4e4c0b1490 2023-07-06T02:23:40.361 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e3006ace0 msgr2=0x7f4e3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e3006ace0 0x7f4e3006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c007fd0 tx=0x7f4e3c007de0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c0b1490 msgr2=0x7f4e4c14fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c0b1490 0x7f4e4c14fcd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800b530 tx=0x7f4e4800b840 comp rx=0 tx=0).stop 2023-07-06T02:23:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 shutdown_connections 2023-07-06T02:23:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4c15c690 0x7f4e4c14b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4c06bfa0 0x7f4e4c0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e3006ace0 0x7f4e3006d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 --2- 172.21.15.89:0/3025025416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4c0b1490 0x7f4e4c14fcd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 >> 172.21.15.89:0/3025025416 conn(0x7f4e4c0cc490 msgr2=0x7f4e4c06a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:40.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 shutdown_connections 2023-07-06T02:23:40.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.361+0000 7f4e54025700 1 -- 172.21.15.89:0/3025025416 wait complete. 2023-07-06T02:23:40.374 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:40.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.501+0000 7f7434256700 1 Processor -- start 2023-07-06T02:23:40.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.501+0000 7f7434256700 1 -- start start 2023-07-06T02:23:40.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.502+0000 7f7434256700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f742c0cd4d0 0x7f742c160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.502+0000 7f7434256700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c0cd930 con 0x7f742c06c5e0 2023-07-06T02:23:40.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.502+0000 7f7434256700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c0cdb60 con 0x7f742c159fd0 2023-07-06T02:23:40.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.502+0000 7f7434256700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742c15ce60 con 0x7f742c0cd4d0 2023-07-06T02:23:40.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.502+0000 7f74317f1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f742c06c5e0 0x7f742c0d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58532/0 (socket says 172.21.15.89:58532) 2023-07-06T02:23:40.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.502+0000 7f74317f1700 1 -- 172.21.15.89:0/158276289 learned_addr learned my addr 172.21.15.89:0/158276289 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:40.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.502+0000 7f74327f3700 1 --2- 172.21.15.89:0/158276289 >> v2:172.21.15.172:3300/0 conn(0x7f742c0cd4d0 0x7f742c160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591636831 0 0) 0x7f742c0cd930 con 0x7f742c06c5e0 2023-07-06T02:23:40.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74180035b0 con 0x7f742c06c5e0 2023-07-06T02:23:40.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3386388204 0 0) 0x7f74180035b0 con 0x7f742c06c5e0 2023-07-06T02:23:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 >> v2:172.21.15.172:3300/0 conn(0x7f742c0cd4d0 msgr2=0x7f742c160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 --2- 172.21.15.89:0/158276289 >> v2:172.21.15.172:3300/0 conn(0x7f742c0cd4d0 0x7f742c160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 >> v1:172.21.15.172:6789/0 conn(0x7f742c159fd0 legacy=0x7f742c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c15d140 con 0x7f742c06c5e0 2023-07-06T02:23:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f7428001670 con 0x7f742c06c5e0 2023-07-06T02:23:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 >> v1:172.21.15.89:6789/0 conn(0x7f742c06c5e0 legacy=0x7f742c0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f7418003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7418004850 0x7f7418004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7418005200 0x7f741800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c15ce60 con 0x7f7418003a00 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741800c080 con 0x7f7418005200 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741800c240 con 0x7f7418004850 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7431ff2700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f7418003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f74317f1700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7418004850 0x7f7418004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f74327f3700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7418005200 0x7f741800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7431ff2700 1 -- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7418004850 msgr2=0x7f7418004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7431ff2700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7418004850 0x7f7418004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7431ff2700 1 -- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7418005200 msgr2=0x7f741800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7431ff2700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7418005200 0x7f741800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f74327f3700 1 -- 172.21.15.89:0/158276289 reap_dead start 2023-07-06T02:23:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7431ff2700 1 -- 172.21.15.89:0/158276289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74180035b0 con 0x7f7418003a00 2023-07-06T02:23:40.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.503+0000 7f7431ff2700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f7418003e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f741c0017c0 tx=0x7f7418005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb70e98ef6550f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0052d0 con 0x7f7418003a00 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c005450 con 0x7f7418003a00 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7430ff0700 1 -- 172.21.15.89:0/158276289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0038c0 con 0x7f7418003a00 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7434256700 1 -- 172.21.15.89:0/158276289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 msgr2=0x7f7418003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7434256700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f7418003e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f741c0017c0 tx=0x7f7418005880 comp rx=0 tx=0).stop 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7434256700 1 -- 172.21.15.89:0/158276289 shutdown_connections 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7434256700 1 --2- 172.21.15.89:0/158276289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f7418003e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7434256700 1 -- 172.21.15.89:0/158276289 >> 172.21.15.89:0/158276289 conn(0x7f742c0d0590 msgr2=0x7f742c0d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:40.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7434256700 1 -- 172.21.15.89:0/158276289 shutdown_connections 2023-07-06T02:23:40.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.504+0000 7f7434256700 1 -- 172.21.15.89:0/158276289 wait complete. 2023-07-06T02:23:40.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.505+0000 7f7434256700 1 Processor -- start 2023-07-06T02:23:40.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.505+0000 7f7434256700 1 -- start start 2023-07-06T02:23:40.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7434256700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f742c0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7434256700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f742c14a370 0x7f742c0b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7434256700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f742c0ca930 0x7f742c0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7434256700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0b8c20 con 0x7f7418003a00 2023-07-06T02:23:40.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7434256700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c14a7e0 con 0x7f742c14a370 2023-07-06T02:23:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7434256700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0b5e50 con 0x7f742c0ca930 2023-07-06T02:23:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7431ff2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f742c0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7431ff2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f742c0c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33642/0 (socket says 172.21.15.89:33642) 2023-07-06T02:23:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7431ff2700 1 -- 172.21.15.89:0/3366508300 learned_addr learned my addr 172.21.15.89:0/3366508300 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7431ff2700 1 -- 172.21.15.89:0/3366508300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f742c0ca930 msgr2=0x7f742c0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7431ff2700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f742c0ca930 0x7f742c0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7431ff2700 1 -- 172.21.15.89:0/3366508300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f742c14a370 msgr2=0x7f742c0b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.506+0000 7f7431ff2700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f742c14a370 0x7f742c0b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.507+0000 7f7431ff2700 1 -- 172.21.15.89:0/3366508300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c0b6130 con 0x7f7418003a00 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.507+0000 7f7431ff2700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f742c0c76c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f741c000c00 tx=0x7f741c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.507+0000 7f74227fc700 1 -- 172.21.15.89:0/3366508300 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c001290 con 0x7f7418003a00 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.507+0000 7f74227fc700 1 -- 172.21.15.89:0/3366508300 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c007040 con 0x7f7418003a00 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.507+0000 7f74227fc700 1 -- 172.21.15.89:0/3366508300 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0073f0 con 0x7f7418003a00 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.507+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c0b6410 con 0x7f7418003a00 2023-07-06T02:23:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.507+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c0b88e0 con 0x7f7418003a00 2023-07-06T02:23:40.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.508+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c0b6930 con 0x7f7418003a00 2023-07-06T02:23:40.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.508+0000 7f74227fc700 1 -- 172.21.15.89:0/3366508300 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f741c007550 con 0x7f7418003a00 2023-07-06T02:23:40.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.508+0000 7f74227fc700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f741006ad10 0x7f741006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.509+0000 7f74227fc700 1 -- 172.21.15.89:0/3366508300 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f741c08a140 con 0x7f7418003a00 2023-07-06T02:23:40.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.509+0000 7f74317f1700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f741006ad10 0x7f741006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.509+0000 7f74317f1700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f741006ad10 0x7f741006d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7428001200 tx=0x7f7428001150 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.513+0000 7f74227fc700 1 -- 172.21.15.89:0/3366508300 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f742c0b6930 con 0x7f7418003a00 2023-07-06T02:23:40.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.703+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f742c0b6930 con 0x7f7418003a00 2023-07-06T02:23:40.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.703+0000 7f74227fc700 1 -- 172.21.15.89:0/3366508300 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f742c0b6930 con 0x7f7418003a00 2023-07-06T02:23:40.705 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:40.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f741006ad10 msgr2=0x7f741006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f741006ad10 0x7f741006d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7428001200 tx=0x7f7428001150 comp rx=0 tx=0).stop 2023-07-06T02:23:40.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 msgr2=0x7f742c0c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f742c0c76c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f741c000c00 tx=0x7f741c001120 comp rx=0 tx=0).stop 2023-07-06T02:23:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 shutdown_connections 2023-07-06T02:23:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f742c0ca930 0x7f742c0c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f741006ad10 0x7f741006d1c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7418003a00 0x7f742c0c76c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 --2- 172.21.15.89:0/3366508300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f742c14a370 0x7f742c0b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 >> 172.21.15.89:0/3366508300 conn(0x7f742c0d0590 msgr2=0x7f742c15e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:40.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 shutdown_connections 2023-07-06T02:23:40.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.705+0000 7f7434256700 1 -- 172.21.15.89:0/3366508300 wait complete. 2023-07-06T02:23:40.719 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.839+0000 7f216bacc700 1 Processor -- start 2023-07-06T02:23:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.839+0000 7f216bacc700 1 -- start start 2023-07-06T02:23:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.840+0000 7f216bacc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.840+0000 7f216bacc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f216406bfa0 0x7f216406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.840+0000 7f216bacc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21640cab50 con 0x7f216415c690 2023-07-06T02:23:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.840+0000 7f216bacc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216415de50 con 0x7f216406bfa0 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.840+0000 7f216bacc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21640b1e20 con 0x7f21640b1490 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.840+0000 7f2169868700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f216415c690 0x7f21640cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58552/0 (socket says 172.21.15.89:58552) 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.840+0000 7f2169868700 1 -- 172.21.15.89:0/1904913453 learned_addr learned my addr 172.21.15.89:0/1904913453 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f2169067700 1 --2- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f216a069700 1 --2- 172.21.15.89:0/1904913453 >> v2:172.21.15.172:3300/0 conn(0x7f216406bfa0 0x7f216406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f2168866700 1 -- 172.21.15.89:0/1904913453 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424070236 0 0) 0x7f21640cab50 con 0x7f216415c690 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f2168866700 1 -- 172.21.15.89:0/1904913453 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21500035b0 con 0x7f216415c690 2023-07-06T02:23:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f216a069700 1 -- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 msgr2=0x7f21640b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f216a069700 1 --2- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f216a069700 1 -- 172.21.15.89:0/1904913453 >> v1:172.21.15.89:6789/0 conn(0x7f216415c690 legacy=0x7f21640cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f2168866700 1 -- 172.21.15.89:0/1904913453 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 555291837 0 0) 0x7f21500035b0 con 0x7f216415c690 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f216a069700 1 -- 172.21.15.89:0/1904913453 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216406c920 con 0x7f216406bfa0 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f216a069700 1 --2- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 0x7f216406c3e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f215c009d90 tx=0x7f215c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37820d2118506ac6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f2168866700 1 -- 172.21.15.89:0/1904913453 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00e070 con 0x7f216406bfa0 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f2168866700 1 -- 172.21.15.89:0/1904913453 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c00bb20 con 0x7f216406bfa0 2023-07-06T02:23:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.841+0000 7f2168866700 1 -- 172.21.15.89:0/1904913453 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00be90 con 0x7f216406bfa0 2023-07-06T02:23:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 -- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 msgr2=0x7f216406c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 --2- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 0x7f216406c3e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f215c009d90 tx=0x7f215c00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 -- 172.21.15.89:0/1904913453 shutdown_connections 2023-07-06T02:23:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 --2- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 0x7f216406c3e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 --2- 172.21.15.89:0/1904913453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 -- 172.21.15.89:0/1904913453 >> 172.21.15.89:0/1904913453 conn(0x7f21640cc490 msgr2=0x7f21640cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 -- 172.21.15.89:0/1904913453 shutdown_connections 2023-07-06T02:23:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.842+0000 7f216bacc700 1 -- 172.21.15.89:0/1904913453 wait complete. 2023-07-06T02:23:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.843+0000 7f216bacc700 1 Processor -- start 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.843+0000 7f216bacc700 1 -- start start 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216bacc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 0x7f21640abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216bacc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216bacc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216415c690 0x7f2164152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216bacc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21640c4780 con 0x7f216415c690 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216bacc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21640c3b00 con 0x7f216406bfa0 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216bacc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21640c3de0 con 0x7f21640b1490 2023-07-06T02:23:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216415c690 0x7f2164152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216415c690 0x7f2164152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33660/0 (socket says 172.21.15.89:33660) 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 -- 172.21.15.89:0/158137379 learned_addr learned my addr 172.21.15.89:0/158137379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f2169067700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 -- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 msgr2=0x7f21640c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f2169868700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 0x7f21640abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 -- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 msgr2=0x7f21640abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 0x7f21640abfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 -- 172.21.15.89:0/158137379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c009a70 con 0x7f216415c690 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.844+0000 7f216a069700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216415c690 0x7f2164152f50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f215c014040 tx=0x7f215c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.845+0000 7f215a7fc700 1 -- 172.21.15.89:0/158137379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0098f0 con 0x7f216415c690 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.845+0000 7f215a7fc700 1 -- 172.21.15.89:0/158137379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c020070 con 0x7f216415c690 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.845+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21640c40c0 con 0x7f216415c690 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.845+0000 7f215a7fc700 1 -- 172.21.15.89:0/158137379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0276a0 con 0x7f216415c690 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.845+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21640ac510 con 0x7f216415c690 2023-07-06T02:23:40.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.846+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21640c48c0 con 0x7f216415c690 2023-07-06T02:23:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.846+0000 7f215a7fc700 1 -- 172.21.15.89:0/158137379 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f215c012430 con 0x7f216415c690 2023-07-06T02:23:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.846+0000 7f215a7fc700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f214806ace0 0x7f214806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:40.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.847+0000 7f215a7fc700 1 -- 172.21.15.89:0/158137379 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f215c022070 con 0x7f216415c690 2023-07-06T02:23:40.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.847+0000 7f2169868700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f214806ace0 0x7f214806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:40.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.847+0000 7f2169868700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f214806ace0 0x7f214806d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2154007fd0 tx=0x7f2154009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:40.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:40.851+0000 7f215a7fc700 1 -- 172.21.15.89:0/158137379 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21640c48c0 con 0x7f216415c690 2023-07-06T02:23:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.038+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21640c48c0 con 0x7f216415c690 2023-07-06T02:23:41.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.038+0000 7f215a7fc700 1 -- 172.21.15.89:0/158137379 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21640c48c0 con 0x7f216415c690 2023-07-06T02:23:41.041 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f214806ace0 msgr2=0x7f214806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f214806ace0 0x7f214806d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2154007fd0 tx=0x7f2154009040 comp rx=0 tx=0).stop 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216415c690 msgr2=0x7f2164152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216415c690 0x7f2164152f50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f215c014040 tx=0x7f215c004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 shutdown_connections 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f214806ace0 0x7f214806d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f216415c690 0x7f2164152f50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216406bfa0 0x7f21640abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 --2- 172.21.15.89:0/158137379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21640b1490 0x7f21640c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 >> 172.21.15.89:0/158137379 conn(0x7f21640cc490 msgr2=0x7f21640cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 shutdown_connections 2023-07-06T02:23:41.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.040+0000 7f216bacc700 1 -- 172.21.15.89:0/158137379 wait complete. 2023-07-06T02:23:41.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.175+0000 7f198364c700 1 Processor -- start 2023-07-06T02:23:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.175+0000 7f198364c700 1 -- start start 2023-07-06T02:23:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f198364c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 0x7f197c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f198364c700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0cb3d0 con 0x7f197c0b19d0 2023-07-06T02:23:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f198364c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0b1df0 con 0x7f197c156ab0 2023-07-06T02:23:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f198364c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c156e80 con 0x7f197c1600b0 2023-07-06T02:23:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1981be9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f197c0b19d0 0x7f197c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58570/0 (socket says 172.21.15.89:58570) 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1981be9700 1 -- 172.21.15.89:0/85833640 learned_addr learned my addr 172.21.15.89:0/85833640 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190205399 0 0) 0x7f197c0cb3d0 con 0x7f197c0b19d0 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19680035b0 con 0x7f197c0b19d0 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1980be7700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 0x7f197c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3800154540 0 0) 0x7f19680035b0 con 0x7f197c0b19d0 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 msgr2=0x7f197c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 0x7f197c0b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 >> v1:172.21.15.172:6789/0 conn(0x7f197c156ab0 legacy=0x7f197c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0ca370 con 0x7f197c0b19d0 2023-07-06T02:23:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.176+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f19740019f0 con 0x7f197c0b19d0 2023-07-06T02:23:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 >> v1:172.21.15.89:6789/0 conn(0x7f197c0b19d0 legacy=0x7f197c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 0x7f1968003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f1968004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1968004d90 0x7f1968005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c156e80 con 0x7f1968003fb0 2023-07-06T02:23:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196800baf0 con 0x7f1968004d90 2023-07-06T02:23:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196800bcb0 con 0x7f197c1600b0 2023-07-06T02:23:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1980be7700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f1968004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f19813e8700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 0x7f1968003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1981be9700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1968004d90 0x7f1968005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1980be7700 1 -- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 msgr2=0x7f1968003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1980be7700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c1600b0 0x7f1968003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1980be7700 1 -- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1968004d90 msgr2=0x7f1968005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1980be7700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1968004d90 0x7f1968005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1980be7700 1 -- 172.21.15.89:0/85833640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680035b0 con 0x7f1968003fb0 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1980be7700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f1968004850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f196c009b30 tx=0x7f1968005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b166716233a4af7c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c00e6c0 con 0x7f1968003fb0 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c00e820 con 0x7f1968003fb0 2023-07-06T02:23:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.177+0000 7f1973fff700 1 -- 172.21.15.89:0/85833640 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c00eb90 con 0x7f1968003fb0 2023-07-06T02:23:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.178+0000 7f198364c700 1 -- 172.21.15.89:0/85833640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 msgr2=0x7f1968004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.178+0000 7f198364c700 1 --2- 172.21.15.89:0/85833640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f1968004850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f196c009b30 tx=0x7f1968005880 comp rx=0 tx=0).stop 2023-07-06T02:23:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.178+0000 7f1981be9700 1 -- 172.21.15.89:0/85833640 reap_dead start 2023-07-06T02:23:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.178+0000 7f198364c700 1 -- 172.21.15.89:0/85833640 shutdown_connections 2023-07-06T02:23:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.178+0000 7f198364c700 1 -- 172.21.15.89:0/85833640 >> 172.21.15.89:0/85833640 conn(0x7f197c0cc490 msgr2=0x7f197c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.178+0000 7f198364c700 1 -- 172.21.15.89:0/85833640 shutdown_connections 2023-07-06T02:23:41.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.178+0000 7f198364c700 1 -- 172.21.15.89:0/85833640 wait complete. 2023-07-06T02:23:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.179+0000 7f198364c700 1 Processor -- start 2023-07-06T02:23:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.179+0000 7f198364c700 1 -- start start 2023-07-06T02:23:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f198364c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f197c0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f198364c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f197c154310 0x7f197c0c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f198364c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c0af470 0x7f197c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f198364c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0afe50 con 0x7f1968003fb0 2023-07-06T02:23:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f198364c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1f2250 con 0x7f197c154310 2023-07-06T02:23:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f198364c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0c6370 con 0x7f197c0af470 2023-07-06T02:23:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f197c0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f197c0c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33676/0 (socket says 172.21.15.89:33676) 2023-07-06T02:23:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 -- 172.21.15.89:0/3069496911 learned_addr learned my addr 172.21.15.89:0/3069496911 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f1980be7700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f197c154310 0x7f197c0c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f1981be9700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c0af470 0x7f197c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 -- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c0af470 msgr2=0x7f197c0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c0af470 0x7f197c0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 -- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f197c154310 msgr2=0x7f197c0c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f197c154310 0x7f197c0c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 -- 172.21.15.89:0/3069496911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0c5ad0 con 0x7f1968003fb0 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.180+0000 7f19813e8700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f197c0c4f10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f197800f7f0 tx=0x7f197800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.181+0000 7f1971ffb700 1 -- 172.21.15.89:0/3069496911 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780052e0 con 0x7f1968003fb0 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.181+0000 7f1971ffb700 1 -- 172.21.15.89:0/3069496911 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978005440 con 0x7f1968003fb0 2023-07-06T02:23:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.181+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c0c5db0 con 0x7f1968003fb0 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.181+0000 7f1971ffb700 1 -- 172.21.15.89:0/3069496911 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780038c0 con 0x7f1968003fb0 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.181+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c0b2990 con 0x7f1968003fb0 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.182+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c0b2b50 con 0x7f1968003fb0 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.182+0000 7f1971ffb700 1 -- 172.21.15.89:0/3069496911 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1978003a20 con 0x7f1968003fb0 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.182+0000 7f1971ffb700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f196006ace0 0x7f196006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.183+0000 7f1971ffb700 1 -- 172.21.15.89:0/3069496911 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1978089560 con 0x7f1968003fb0 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.183+0000 7f1980be7700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f196006ace0 0x7f196006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.183+0000 7f1980be7700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f196006ace0 0x7f196006d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f196c0090b0 tx=0x7f196c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.187+0000 7f1971ffb700 1 -- 172.21.15.89:0/3069496911 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1978054370 con 0x7f1968003fb0 2023-07-06T02:23:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.377+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f197c074270 con 0x7f1968003fb0 2023-07-06T02:23:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.378+0000 7f1971ffb700 1 -- 172.21.15.89:0/3069496911 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1978057990 con 0x7f1968003fb0 2023-07-06T02:23:41.380 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f196006ace0 msgr2=0x7f196006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f196006ace0 0x7f196006d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f196c0090b0 tx=0x7f196c008030 comp rx=0 tx=0).stop 2023-07-06T02:23:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 msgr2=0x7f197c0c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f197c0c4f10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f197800f7f0 tx=0x7f197800fbb0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 shutdown_connections 2023-07-06T02:23:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f197c0af470 0x7f197c0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f196006ace0 0x7f196006d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1968003fb0 0x7f197c0c4f10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 --2- 172.21.15.89:0/3069496911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f197c154310 0x7f197c0c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 >> 172.21.15.89:0/3069496911 conn(0x7f197c0cc490 msgr2=0x7f197c160480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 shutdown_connections 2023-07-06T02:23:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.380+0000 7f198364c700 1 -- 172.21.15.89:0/3069496911 wait complete. 2023-07-06T02:23:41.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.518+0000 7f51e7f19700 1 Processor -- start 2023-07-06T02:23:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.518+0000 7f51e7f19700 1 -- start start 2023-07-06T02:23:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e7f19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51e00cc000 0x7f51e00caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e7f19700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f51e006b220 0x7f51e015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e7f19700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e006a990 con 0x7f51e00b1490 2023-07-06T02:23:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e7f19700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e006ac70 con 0x7f51e00cc000 2023-07-06T02:23:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e7f19700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e006d170 con 0x7f51e006b220 2023-07-06T02:23:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e54b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51e00cc000 0x7f51e00caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e5cb5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f51e00b1490 0x7f51e00b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58586/0 (socket says 172.21.15.89:58586) 2023-07-06T02:23:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e54b4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51e00cc000 0x7f51e00caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33686/0 (socket says 172.21.15.89:33686) 2023-07-06T02:23:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e5cb5700 1 -- 172.21.15.89:0/257524070 learned_addr learned my addr 172.21.15.89:0/257524070 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e64b6700 1 --2- 172.21.15.89:0/257524070 >> v2:172.21.15.172:3300/0 conn(0x7f51e006b220 0x7f51e015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e4cb3700 1 -- 172.21.15.89:0/257524070 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324828228 0 0) 0x7f51e006a990 con 0x7f51e00b1490 2023-07-06T02:23:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e4cb3700 1 -- 172.21.15.89:0/257524070 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51cc0035b0 con 0x7f51e00b1490 2023-07-06T02:23:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e54b4700 1 -- 172.21.15.89:0/257524070 >> v2:172.21.15.172:3300/0 conn(0x7f51e006b220 msgr2=0x7f51e015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e54b4700 1 --2- 172.21.15.89:0/257524070 >> v2:172.21.15.172:3300/0 conn(0x7f51e006b220 0x7f51e015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e54b4700 1 -- 172.21.15.89:0/257524070 >> v1:172.21.15.89:6789/0 conn(0x7f51e00b1490 legacy=0x7f51e00b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e54b4700 1 -- 172.21.15.89:0/257524070 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e006ce60 con 0x7f51e00cc000 2023-07-06T02:23:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.519+0000 7f51e54b4700 1 --2- 172.21.15.89:0/257524070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00cc000 0x7f51e00caeb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51e00b2460 tx=0x7f51d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2c29049bc3d943 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e4cb3700 1 -- 172.21.15.89:0/257524070 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0011830 con 0x7f51e00cc000 2023-07-06T02:23:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e4cb3700 1 -- 172.21.15.89:0/257524070 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0011990 con 0x7f51e00cc000 2023-07-06T02:23:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e4cb3700 1 -- 172.21.15.89:0/257524070 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0011d00 con 0x7f51e00cc000 2023-07-06T02:23:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 -- 172.21.15.89:0/257524070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00cc000 msgr2=0x7f51e00caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 --2- 172.21.15.89:0/257524070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00cc000 0x7f51e00caeb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51e00b2460 tx=0x7f51d0009b20 comp rx=0 tx=0).stop 2023-07-06T02:23:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 -- 172.21.15.89:0/257524070 shutdown_connections 2023-07-06T02:23:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 --2- 172.21.15.89:0/257524070 >> v2:172.21.15.172:3300/0 conn(0x7f51e006b220 0x7f51e015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 --2- 172.21.15.89:0/257524070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00cc000 0x7f51e00caeb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 -- 172.21.15.89:0/257524070 >> 172.21.15.89:0/257524070 conn(0x7f51e00cc490 msgr2=0x7f51e00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 -- 172.21.15.89:0/257524070 shutdown_connections 2023-07-06T02:23:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.520+0000 7f51e7f19700 1 -- 172.21.15.89:0/257524070 wait complete. 2023-07-06T02:23:41.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.521+0000 7f51e7f19700 1 Processor -- start 2023-07-06T02:23:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.521+0000 7f51e7f19700 1 -- start start 2023-07-06T02:23:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e7f19700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e006b220 0x7f51e0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e7f19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00b1490 0x7f51e01e4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e7f19700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51e00cc000 0x7f51e00c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e7f19700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e01e57e0 con 0x7f51e00b1490 2023-07-06T02:23:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e7f19700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e015b6b0 con 0x7f51e006b220 2023-07-06T02:23:41.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e7f19700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e015b380 con 0x7f51e00cc000 2023-07-06T02:23:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00b1490 0x7f51e01e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00b1490 0x7f51e01e4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33692/0 (socket says 172.21.15.89:33692) 2023-07-06T02:23:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 -- 172.21.15.89:0/3601763660 learned_addr learned my addr 172.21.15.89:0/3601763660 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e5cb5700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e006b220 0x7f51e0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 -- 172.21.15.89:0/3601763660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51e00cc000 msgr2=0x7f51e00c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51e00cc000 0x7f51e00c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 -- 172.21.15.89:0/3601763660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e006b220 msgr2=0x7f51e0151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e006b220 0x7f51e0151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.522+0000 7f51e54b4700 1 -- 172.21.15.89:0/3601763660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00094a0 con 0x7f51e00b1490 2023-07-06T02:23:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.523+0000 7f51e54b4700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00b1490 0x7f51e01e4640 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c00 tx=0x7f51d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.523+0000 7f51d67fc700 1 -- 172.21.15.89:0/3601763660 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00045d0 con 0x7f51e00b1490 2023-07-06T02:23:41.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.523+0000 7f51d67fc700 1 -- 172.21.15.89:0/3601763660 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d001f070 con 0x7f51e00b1490 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.523+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e01e4f40 con 0x7f51e00b1490 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.523+0000 7f51d67fc700 1 -- 172.21.15.89:0/3601763660 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00038a0 con 0x7f51e00b1490 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.523+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e01e5500 con 0x7f51e00b1490 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.524+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e0074270 con 0x7f51e00b1490 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.524+0000 7f51d67fc700 1 -- 172.21.15.89:0/3601763660 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51d0003a00 con 0x7f51e00b1490 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.525+0000 7f51d67fc700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51c406ace0 0x7f51c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.525+0000 7f51d67fc700 1 -- 172.21.15.89:0/3601763660 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f51d0021070 con 0x7f51e00b1490 2023-07-06T02:23:41.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.525+0000 7f51e5cb5700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51c406ace0 0x7f51c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.525+0000 7f51e5cb5700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51c406ace0 0x7f51c406d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f60 tx=0x7f51dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.529+0000 7f51d67fc700 1 -- 172.21.15.89:0/3601763660 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51d0055d80 con 0x7f51e00b1490 2023-07-06T02:23:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.720+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51e015b7f0 con 0x7f51e00b1490 2023-07-06T02:23:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.721+0000 7f51d67fc700 1 -- 172.21.15.89:0/3601763660 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51d002b0a0 con 0x7f51e00b1490 2023-07-06T02:23:41.723 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:41.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51c406ace0 msgr2=0x7f51c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51c406ace0 0x7f51c406d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f60 tx=0x7f51dc009040 comp rx=0 tx=0).stop 2023-07-06T02:23:41.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00b1490 msgr2=0x7f51e01e4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00b1490 0x7f51e01e4640 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c00 tx=0x7f51d0004750 comp rx=0 tx=0).stop 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 shutdown_connections 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51e00cc000 0x7f51e00c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e006b220 0x7f51e0151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51c406ace0 0x7f51c406d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 --2- 172.21.15.89:0/3601763660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51e00b1490 0x7f51e01e4640 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 >> 172.21.15.89:0/3601763660 conn(0x7f51e00cc490 msgr2=0x7f51e006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 shutdown_connections 2023-07-06T02:23:41.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.723+0000 7f51e7f19700 1 -- 172.21.15.89:0/3601763660 wait complete. 2023-07-06T02:23:41.783 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-06T02:23:41.784 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-07-06T02:23:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.904+0000 7f8f44613700 1 Processor -- start 2023-07-06T02:23:41.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.904+0000 7f8f44613700 1 -- start start 2023-07-06T02:23:41.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f44613700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f3c0b1ba0 0x7f8f3c167340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f44613700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0cb3d0 con 0x7f8f3c1600b0 2023-07-06T02:23:41.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f44613700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c156e80 con 0x7f8f3c156ab0 2023-07-06T02:23:41.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f44613700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c0ca370 con 0x7f8f3c0b1ba0 2023-07-06T02:23:41.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f41bae700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8f3c1600b0 0x7f8f3c0b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58600/0 (socket says 172.21.15.89:58600) 2023-07-06T02:23:41.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f41bae700 1 -- 172.21.15.89:0/246633778 learned_addr learned my addr 172.21.15.89:0/246633778 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:41.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f42bb0700 1 --2- 172.21.15.89:0/246633778 >> v2:172.21.15.89:3300/0 conn(0x7f8f3c0b1ba0 0x7f8f3c167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f42bb0700 1 -- 172.21.15.89:0/246633778 >> v1:172.21.15.172:6789/0 conn(0x7f8f3c156ab0 legacy=0x7f8f3c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:41.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f42bb0700 1 -- 172.21.15.89:0/246633778 >> v1:172.21.15.89:6789/0 conn(0x7f8f3c1600b0 legacy=0x7f8f3c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f413ad700 1 -- 172.21.15.89:0/246633778 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198497941 0 0) 0x7f8f3c0cb3d0 con 0x7f8f3c1600b0 2023-07-06T02:23:41.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f42bb0700 1 -- 172.21.15.89:0/246633778 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c160d80 con 0x7f8f3c0b1ba0 2023-07-06T02:23:41.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f42bb0700 1 --2- 172.21.15.89:0/246633778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c0b1ba0 0x7f8f3c167340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009d90 tx=0x7f8f3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d67671b69a74043 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f413ad700 1 -- 172.21.15.89:0/246633778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400e070 con 0x7f8f3c0b1ba0 2023-07-06T02:23:41.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f413ad700 1 -- 172.21.15.89:0/246633778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400bb90 con 0x7f8f3c0b1ba0 2023-07-06T02:23:41.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.905+0000 7f8f413ad700 1 -- 172.21.15.89:0/246633778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003a20 con 0x7f8f3c0b1ba0 2023-07-06T02:23:41.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.906+0000 7f8f44613700 1 -- 172.21.15.89:0/246633778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c0b1ba0 msgr2=0x7f8f3c167340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.906+0000 7f8f44613700 1 --2- 172.21.15.89:0/246633778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c0b1ba0 0x7f8f3c167340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009d90 tx=0x7f8f3400b330 comp rx=0 tx=0).stop 2023-07-06T02:23:41.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.906+0000 7f8f44613700 1 -- 172.21.15.89:0/246633778 shutdown_connections 2023-07-06T02:23:41.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.906+0000 7f8f44613700 1 --2- 172.21.15.89:0/246633778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c0b1ba0 0x7f8f3c167340 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.906+0000 7f8f44613700 1 -- 172.21.15.89:0/246633778 >> 172.21.15.89:0/246633778 conn(0x7f8f3c0cc490 msgr2=0x7f8f3c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:41.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.906+0000 7f8f44613700 1 -- 172.21.15.89:0/246633778 shutdown_connections 2023-07-06T02:23:41.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.906+0000 7f8f44613700 1 -- 172.21.15.89:0/246633778 wait complete. 2023-07-06T02:23:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.907+0000 7f8f44613700 1 Processor -- start 2023-07-06T02:23:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.907+0000 7f8f44613700 1 -- start start 2023-07-06T02:23:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f44613700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f3c0b1ba0 0x7f8f3c0acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f44613700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f3c156ab0 0x7f8f3c154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f44613700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c1600b0 0x7f8f3c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f44613700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0ac6a0 con 0x7f8f3c1600b0 2023-07-06T02:23:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f44613700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0aba20 con 0x7f8f3c156ab0 2023-07-06T02:23:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f44613700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0abd00 con 0x7f8f3c0b1ba0 2023-07-06T02:23:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c1600b0 0x7f8f3c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c1600b0 0x7f8f3c0b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33708/0 (socket says 172.21.15.89:33708) 2023-07-06T02:23:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 -- 172.21.15.89:0/3723339533 learned_addr learned my addr 172.21.15.89:0/3723339533 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f41bae700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f3c156ab0 0x7f8f3c154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 -- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f3c0b1ba0 msgr2=0x7f8f3c0acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f423af700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f3c0b1ba0 0x7f8f3c0acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f3c0b1ba0 0x7f8f3c0acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 -- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f3c156ab0 msgr2=0x7f8f3c154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f3c156ab0 0x7f8f3c154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 -- 172.21.15.89:0/3723339533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009a70 con 0x7f8f3c1600b0 2023-07-06T02:23:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f42bb0700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c1600b0 0x7f8f3c0b5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0c3510 tx=0x7f8f34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f32ffd700 1 -- 172.21.15.89:0/3723339533 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34012630 con 0x7f8f3c1600b0 2023-07-06T02:23:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.908+0000 7f8f32ffd700 1 -- 172.21.15.89:0/3723339533 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f340127b0 con 0x7f8f3c1600b0 2023-07-06T02:23:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.909+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c0abfe0 con 0x7f8f3c1600b0 2023-07-06T02:23:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.909+0000 7f8f32ffd700 1 -- 172.21.15.89:0/3723339533 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34012b60 con 0x7f8f3c1600b0 2023-07-06T02:23:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.909+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c155310 con 0x7f8f3c1600b0 2023-07-06T02:23:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.909+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c074270 con 0x7f8f3c1600b0 2023-07-06T02:23:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.910+0000 7f8f32ffd700 1 -- 172.21.15.89:0/3723339533 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f34012cc0 con 0x7f8f3c1600b0 2023-07-06T02:23:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.910+0000 7f8f32ffd700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8f2006ace0 0x7f8f2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.910+0000 7f8f32ffd700 1 -- 172.21.15.89:0/3723339533 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8f3408bb90 con 0x7f8f3c1600b0 2023-07-06T02:23:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.910+0000 7f8f423af700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8f2006ace0 0x7f8f2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.911+0000 7f8f423af700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8f2006ace0 0x7f8f2006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f38003e60 tx=0x7f8f38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:41.914+0000 7f8f32ffd700 1 -- 172.21.15.89:0/3723339533 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f3405a430 con 0x7f8f3c1600b0 2023-07-06T02:23:42.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.181+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7f8f3c0cc9e0 con 0x7f8f3c1600b0 2023-07-06T02:23:42.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.402+0000 7f8f32ffd700 1 -- 172.21.15.89:0/3723339533 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v68) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8f34059fc0 con 0x7f8f3c1600b0 2023-07-06T02:23:42.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8f2006ace0 msgr2=0x7f8f2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8f2006ace0 0x7f8f2006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f38003e60 tx=0x7f8f38009040 comp rx=0 tx=0).stop 2023-07-06T02:23:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c1600b0 msgr2=0x7f8f3c0b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c1600b0 0x7f8f3c0b5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0c3510 tx=0x7f8f34004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 shutdown_connections 2023-07-06T02:23:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8f2006ace0 0x7f8f2006d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c1600b0 0x7f8f3c0b5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f3c0b1ba0 0x7f8f3c0acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 --2- 172.21.15.89:0/3723339533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f3c156ab0 0x7f8f3c154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 >> 172.21.15.89:0/3723339533 conn(0x7f8f3c0cc490 msgr2=0x7f8f3c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:42.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 shutdown_connections 2023-07-06T02:23:42.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.405+0000 7f8f44613700 1 -- 172.21.15.89:0/3723339533 wait complete. 2023-07-06T02:23:42.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T02:23:42.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.542+0000 7fe6a0832700 1 Processor -- start 2023-07-06T02:23:42.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.542+0000 7fe6a0832700 1 -- start start 2023-07-06T02:23:42.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.543+0000 7fe6a0832700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c15c610 0x7fe69c0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:42.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.543+0000 7fe6a0832700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe69c0b5e50 0x7fe69c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:42.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.543+0000 7fe6a0832700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c0cb150 con 0x7fe69c070870 2023-07-06T02:23:42.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.543+0000 7fe6a0832700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69c070cf0 con 0x7fe69c0b5e50 2023-07-06T02:23:42.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.543+0000 7fe6a0832700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0cb430 con 0x7fe69c15c610 2023-07-06T02:23:42.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c15c610 0x7fe69c0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:42.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe699d9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe69c0b5e50 0x7fe69c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:42.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c15c610 0x7fe69c0cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:56920/0 (socket says 172.21.15.89:56920) 2023-07-06T02:23:42.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 -- 172.21.15.89:0/1205893408 learned_addr learned my addr 172.21.15.89:0/1205893408 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:42.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 -- 172.21.15.89:0/1205893408 >> v2:172.21.15.172:3300/0 conn(0x7fe69c0b5e50 msgr2=0x7fe69c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:42.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 --2- 172.21.15.89:0/1205893408 >> v2:172.21.15.172:3300/0 conn(0x7fe69c0b5e50 0x7fe69c070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 -- 172.21.15.89:0/1205893408 >> v1:172.21.15.172:6789/0 conn(0x7fe69c070870 legacy=0x7fe69c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 -- 172.21.15.89:0/1205893408 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c0b6310 con 0x7fe69c15c610 2023-07-06T02:23:42.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69a59c700 1 --2- 172.21.15.89:0/1205893408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c15c610 0x7fe69c0cfed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6900098f0 tx=0x7fe690009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62790422e41a86b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:42.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.544+0000 7fe69959a700 1 -- 172.21.15.89:0/1205893408 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690011830 con 0x7fe69c15c610 2023-07-06T02:23:42.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe69959a700 1 -- 172.21.15.89:0/1205893408 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690011990 con 0x7fe69c15c610 2023-07-06T02:23:42.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe69959a700 1 -- 172.21.15.89:0/1205893408 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690011d20 con 0x7fe69c15c610 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 -- 172.21.15.89:0/1205893408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c15c610 msgr2=0x7fe69c0cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1205893408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c15c610 0x7fe69c0cfed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6900098f0 tx=0x7fe690009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 -- 172.21.15.89:0/1205893408 shutdown_connections 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1205893408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c15c610 0x7fe69c0cfed0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1205893408 >> v2:172.21.15.172:3300/0 conn(0x7fe69c0b5e50 0x7fe69c070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 -- 172.21.15.89:0/1205893408 >> 172.21.15.89:0/1205893408 conn(0x7fe69c0d0b00 msgr2=0x7fe69c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 -- 172.21.15.89:0/1205893408 shutdown_connections 2023-07-06T02:23:42.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.545+0000 7fe6a0832700 1 -- 172.21.15.89:0/1205893408 wait complete. 2023-07-06T02:23:42.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.546+0000 7fe6a0832700 1 Processor -- start 2023-07-06T02:23:42.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.546+0000 7fe6a0832700 1 -- start start 2023-07-06T02:23:42.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe6a0832700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe69c070870 0x7fe69c15f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:42.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe6a0832700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c0b5e50 0x7fe69c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:42.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe6a0832700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe69c15c610 0x7fe69c0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:42.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe6a0832700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c14fc80 con 0x7fe69c070870 2023-07-06T02:23:42.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe6a0832700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0c4990 con 0x7fe69c15c610 2023-07-06T02:23:42.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe6a0832700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0c46b0 con 0x7fe69c0b5e50 2023-07-06T02:23:42.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe69a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe69c070870 0x7fe69c15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:42.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe69a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe69c070870 0x7fe69c15f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33718/0 (socket says 172.21.15.89:33718) 2023-07-06T02:23:42.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.547+0000 7fe69a59c700 1 -- 172.21.15.89:0/1087125273 learned_addr learned my addr 172.21.15.89:0/1087125273 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:42.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe69a59c700 1 -- 172.21.15.89:0/1087125273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c0b5e50 msgr2=0x7fe69c14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe69a59c700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c0b5e50 0x7fe69c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe69a59c700 1 -- 172.21.15.89:0/1087125273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe69c15c610 msgr2=0x7fe69c0c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe69a59c700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe69c15c610 0x7fe69c0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe69a59c700 1 -- 172.21.15.89:0/1087125273 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690009550 con 0x7fe69c070870 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe69a59c700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe69c070870 0x7fe69c15f670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe690000c00 tx=0x7fe690004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe68affd700 1 -- 172.21.15.89:0/1087125273 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690004530 con 0x7fe69c070870 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe68affd700 1 -- 172.21.15.89:0/1087125273 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69001f070 con 0x7fe69c070870 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c15ee70 con 0x7fe69c070870 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe68affd700 1 -- 172.21.15.89:0/1087125273 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6900038e0 con 0x7fe69c070870 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.548+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c14f8a0 con 0x7fe69c070870 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.549+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69c15f390 con 0x7fe69c070870 2023-07-06T02:23:42.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.549+0000 7fe68affd700 1 -- 172.21.15.89:0/1087125273 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe690021030 con 0x7fe69c070870 2023-07-06T02:23:42.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.550+0000 7fe68affd700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe67c06ad10 0x7fe67c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:42.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.550+0000 7fe68affd700 1 -- 172.21.15.89:0/1087125273 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe69008aaf0 con 0x7fe69c070870 2023-07-06T02:23:42.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.550+0000 7fe699d9b700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe67c06ad10 0x7fe67c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:42.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.550+0000 7fe699d9b700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe67c06ad10 0x7fe67c06d1c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006b20 tx=0x7fe68c006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:42.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.554+0000 7fe68affd700 1 -- 172.21.15.89:0/1087125273 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe69c15f390 con 0x7fe69c070870 2023-07-06T02:23:42.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.742+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe69c15f390 con 0x7fe69c070870 2023-07-06T02:23:42.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.746+0000 7fe68affd700 1 -- 172.21.15.89:0/1087125273 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe69c15f390 con 0x7fe69c070870 2023-07-06T02:23:42.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe67c06ad10 msgr2=0x7fe67c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe67c06ad10 0x7fe67c06d1c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006b20 tx=0x7fe68c006ab0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe69c070870 msgr2=0x7fe69c15f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:42.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe69c070870 0x7fe69c15f670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe690000c00 tx=0x7fe690004750 comp rx=0 tx=0).stop 2023-07-06T02:23:42.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 shutdown_connections 2023-07-06T02:23:42.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe69c15c610 0x7fe69c0c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe67c06ad10 0x7fe67c06d1c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe69c070870 0x7fe69c15f670 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 --2- 172.21.15.89:0/1087125273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe69c0b5e50 0x7fe69c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:42.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 >> 172.21.15.89:0/1087125273 conn(0x7fe69c0d0b00 msgr2=0x7fe69c15b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:42.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 shutdown_connections 2023-07-06T02:23:42.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:42.748+0000 7fe6a0832700 1 -- 172.21.15.89:0/1087125273 wait complete. 2023-07-06T02:23:42.762 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T02:23:42.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T02:23:42.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T02:23:42.847 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T02:23:42.847 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:42.848 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:42.864 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:42.865 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:42.921 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:42.922 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:42.978 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:43.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.137+0000 7f679f38f700 1 Processor -- start 2023-07-06T02:23:43.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.137+0000 7f679f38f700 1 -- start start 2023-07-06T02:23:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.137+0000 7f679f38f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f679815c500 0x7f679806edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.137+0000 7f679f38f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6798160710 0x7f67980713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.137+0000 7f679f38f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67980cb9f0 con 0x7f6798070f20 2023-07-06T02:23:43.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679f38f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67980718e0 con 0x7f6798160710 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679f38f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679815d8e0 con 0x7f679815c500 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6798160710 0x7f67980713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6798160710 0x7f67980713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33730/0 (socket says 172.21.15.89:33730) 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 -- 172.21.15.89:0/1019286972 learned_addr learned my addr 172.21.15.89:0/1019286972 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 -- 172.21.15.89:0/1019286972 >> v2:172.21.15.172:3300/0 conn(0x7f679815c500 msgr2=0x7f679806edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 --2- 172.21.15.89:0/1019286972 >> v2:172.21.15.172:3300/0 conn(0x7f679815c500 0x7f679806edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 -- 172.21.15.89:0/1019286972 >> v1:172.21.15.172:6789/0 conn(0x7f6798070f20 legacy=0x7f6798160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:43.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 -- 172.21.15.89:0/1019286972 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679806f310 con 0x7f6798160710 2023-07-06T02:23:43.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f679d92c700 1 --2- 172.21.15.89:0/1019286972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6798160710 0x7f67980713a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f679806cd40 tx=0x7f679000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ddbe370c70ce8e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f678ffff700 1 -- 172.21.15.89:0/1019286972 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679000e070 con 0x7f6798160710 2023-07-06T02:23:43.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f678ffff700 1 -- 172.21.15.89:0/1019286972 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790003a20 con 0x7f6798160710 2023-07-06T02:23:43.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.138+0000 7f678ffff700 1 -- 172.21.15.89:0/1019286972 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790003dd0 con 0x7f6798160710 2023-07-06T02:23:43.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 -- 172.21.15.89:0/1019286972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6798160710 msgr2=0x7f67980713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 --2- 172.21.15.89:0/1019286972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6798160710 0x7f67980713a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f679806cd40 tx=0x7f679000b330 comp rx=0 tx=0).stop 2023-07-06T02:23:43.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 -- 172.21.15.89:0/1019286972 shutdown_connections 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 --2- 172.21.15.89:0/1019286972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6798160710 0x7f67980713a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 --2- 172.21.15.89:0/1019286972 >> v2:172.21.15.172:3300/0 conn(0x7f679815c500 0x7f679806edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 -- 172.21.15.89:0/1019286972 >> 172.21.15.89:0/1019286972 conn(0x7f67980cf1c0 msgr2=0x7f67980cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 -- 172.21.15.89:0/1019286972 shutdown_connections 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.139+0000 7f679f38f700 1 -- 172.21.15.89:0/1019286972 wait complete. 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.140+0000 7f679f38f700 1 Processor -- start 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.140+0000 7f679f38f700 1 -- start start 2023-07-06T02:23:43.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679f38f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6798070f20 0x7f67980b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679f38f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679815c500 0x7f67980c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679f38f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6798160710 0x7f67980aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679f38f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67980b1b80 con 0x7f679815c500 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679f38f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67980b18a0 con 0x7f6798160710 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679f38f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67980acae0 con 0x7f6798070f20 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679815c500 0x7f67980c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679815c500 0x7f67980c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33734/0 (socket says 172.21.15.89:33734) 2023-07-06T02:23:43.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 -- 172.21.15.89:0/759410100 learned_addr learned my addr 172.21.15.89:0/759410100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679d92c700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6798160710 0x7f67980aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679d12b700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6798070f20 0x7f67980b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 -- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6798070f20 msgr2=0x7f67980b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6798070f20 0x7f67980b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 -- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6798160710 msgr2=0x7f67980aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6798160710 0x7f67980aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 -- 172.21.15.89:0/759410100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6790009b30 con 0x7f679815c500 2023-07-06T02:23:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679c92a700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679815c500 0x7f67980c5550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f679400c4a0 tx=0x7f679400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f678dffb700 1 -- 172.21.15.89:0/759410100 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67940054e0 con 0x7f679815c500 2023-07-06T02:23:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f678dffb700 1 -- 172.21.15.89:0/759410100 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6794003680 con 0x7f679815c500 2023-07-06T02:23:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67980acdc0 con 0x7f679815c500 2023-07-06T02:23:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.141+0000 7f678dffb700 1 -- 172.21.15.89:0/759410100 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794003a30 con 0x7f679815c500 2023-07-06T02:23:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.142+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67980c5a90 con 0x7f679815c500 2023-07-06T02:23:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.142+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67980ac940 con 0x7f679815c500 2023-07-06T02:23:43.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.143+0000 7f678dffb700 1 -- 172.21.15.89:0/759410100 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6794003b90 con 0x7f679815c500 2023-07-06T02:23:43.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.143+0000 7f678dffb700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f677c06ace0 0x7f677c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.144+0000 7f679d12b700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f677c06ace0 0x7f677c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.144+0000 7f678dffb700 1 -- 172.21.15.89:0/759410100 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6794088d00 con 0x7f679815c500 2023-07-06T02:23:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.144+0000 7f679d12b700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f677c06ace0 0x7f677c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6788006fd0 tx=0x7f6788008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.148+0000 7f678dffb700 1 -- 172.21.15.89:0/759410100 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f67980ac940 con 0x7f679815c500 2023-07-06T02:23:43.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.338+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67980ac940 con 0x7f679815c500 2023-07-06T02:23:43.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.339+0000 7f678dffb700 1 -- 172.21.15.89:0/759410100 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f67980ac940 con 0x7f679815c500 2023-07-06T02:23:43.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:43.341 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:23:41.389931+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:43.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f677c06ace0 msgr2=0x7f677c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f677c06ace0 0x7f677c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6788006fd0 tx=0x7f6788008040 comp rx=0 tx=0).stop 2023-07-06T02:23:43.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679815c500 msgr2=0x7f67980c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679815c500 0x7f67980c5550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f679400c4a0 tx=0x7f679400c860 comp rx=0 tx=0).stop 2023-07-06T02:23:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 shutdown_connections 2023-07-06T02:23:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6798160710 0x7f67980aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6798070f20 0x7f67980b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f677c06ace0 0x7f677c06d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 --2- 172.21.15.89:0/759410100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f679815c500 0x7f67980c5550 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 >> 172.21.15.89:0/759410100 conn(0x7f67980cf1c0 msgr2=0x7f679815a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 shutdown_connections 2023-07-06T02:23:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.341+0000 7f679f38f700 1 -- 172.21.15.89:0/759410100 wait complete. 2023-07-06T02:23:43.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:43.357 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:43.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.480+0000 7f8c43f0b700 1 Processor -- start 2023-07-06T02:23:43.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.480+0000 7f8c43f0b700 1 -- start start 2023-07-06T02:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c43f0b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 0x7f8c3c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c43f0b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c3c0cd180 0x7f8c3c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c43f0b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c06e110 con 0x7f8c3c06dc10 2023-07-06T02:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c43f0b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c06e3f0 con 0x7f8c3c0cd180 2023-07-06T02:23:43.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c43f0b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c06e6d0 con 0x7f8c3c0705a0 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c414a6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c3c0cd180 0x7f8c3c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c424a8700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8c3c06dc10 0x7f8c3c157950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58646/0 (socket says 172.21.15.89:58646) 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c424a8700 1 -- 172.21.15.89:0/1656862234 learned_addr learned my addr 172.21.15.89:0/1656862234 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c414a6700 1 -- 172.21.15.89:0/1656862234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 msgr2=0x7f8c3c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c414a6700 1 --2- 172.21.15.89:0/1656862234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 0x7f8c3c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c414a6700 1 -- 172.21.15.89:0/1656862234 >> v1:172.21.15.89:6789/0 conn(0x7f8c3c06dc10 legacy=0x7f8c3c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c414a6700 1 -- 172.21.15.89:0/1656862234 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c06e9b0 con 0x7f8c3c0cd180 2023-07-06T02:23:43.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c40ca5700 1 -- 172.21.15.89:0/1656862234 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900622523 0 0) 0x7f8c3c06e110 con 0x7f8c3c06dc10 2023-07-06T02:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c414a6700 1 --2- 172.21.15.89:0/1656862234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15a870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009840 tx=0x7f8c2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446ad5d5724f5cfe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c40ca5700 1 -- 172.21.15.89:0/1656862234 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c011830 con 0x7f8c3c0cd180 2023-07-06T02:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c40ca5700 1 -- 172.21.15.89:0/1656862234 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c011990 con 0x7f8c3c0cd180 2023-07-06T02:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.481+0000 7f8c40ca5700 1 -- 172.21.15.89:0/1656862234 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c011d20 con 0x7f8c3c0cd180 2023-07-06T02:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1656862234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 msgr2=0x7f8c3c15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1656862234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15a870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009840 tx=0x7f8c2c009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:43.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1656862234 shutdown_connections 2023-07-06T02:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1656862234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 0x7f8c3c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1656862234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15a870 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1656862234 >> 172.21.15.89:0/1656862234 conn(0x7f8c3c0d0b00 msgr2=0x7f8c3c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1656862234 shutdown_connections 2023-07-06T02:23:43.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.482+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1656862234 wait complete. 2023-07-06T02:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.484+0000 7f8c43f0b700 1 Processor -- start 2023-07-06T02:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.484+0000 7f8c43f0b700 1 -- start start 2023-07-06T02:23:43.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.484+0000 7f8c43f0b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c3c06dc10 0x7f8c3c148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.484+0000 7f8c43f0b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 0x7f8c3c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c43f0b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c43f0b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c154560 con 0x7f8c3c0cd180 2023-07-06T02:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c43f0b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c149460 con 0x7f8c3c06dc10 2023-07-06T02:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c43f0b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c153950 con 0x7f8c3c0705a0 2023-07-06T02:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33750/0 (socket says 172.21.15.89:33750) 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 -- 172.21.15.89:0/1236688649 learned_addr learned my addr 172.21.15.89:0/1236688649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c41ca7700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c3c06dc10 0x7f8c3c148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c414a6700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 0x7f8c3c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 -- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 msgr2=0x7f8c3c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 0x7f8c3c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 -- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c3c06dc10 msgr2=0x7f8c3c148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c3c06dc10 0x7f8c3c148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 -- 172.21.15.89:0/1236688649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0094a0 con 0x7f8c3c0cd180 2023-07-06T02:23:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c424a8700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15b130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c34002230 tx=0x7f8c3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c327fc700 1 -- 172.21.15.89:0/1236688649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400ac20 con 0x7f8c3c0cd180 2023-07-06T02:23:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c327fc700 1 -- 172.21.15.89:0/1236688649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3400ada0 con 0x7f8c3c0cd180 2023-07-06T02:23:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c327fc700 1 -- 172.21.15.89:0/1236688649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34007310 con 0x7f8c3c0cd180 2023-07-06T02:23:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c153c30 con 0x7f8c3c0cd180 2023-07-06T02:23:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.485+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c1541f0 con 0x7f8c3c0cd180 2023-07-06T02:23:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.486+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c1546d0 con 0x7f8c3c0cd180 2023-07-06T02:23:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.487+0000 7f8c327fc700 1 -- 172.21.15.89:0/1236688649 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c34007470 con 0x7f8c3c0cd180 2023-07-06T02:23:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.487+0000 7f8c327fc700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8c2006ad10 0x7f8c2006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.487+0000 7f8c327fc700 1 -- 172.21.15.89:0/1236688649 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8c34089370 con 0x7f8c3c0cd180 2023-07-06T02:23:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.487+0000 7f8c41ca7700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8c2006ad10 0x7f8c2006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.487+0000 7f8c41ca7700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8c2006ad10 0x7f8c2006d1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.491+0000 7f8c327fc700 1 -- 172.21.15.89:0/1236688649 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8c34054060 con 0x7f8c3c0cd180 2023-07-06T02:23:43.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.678+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8c3c074270 con 0x7f8c3c0cd180 2023-07-06T02:23:43.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.679+0000 7f8c327fc700 1 -- 172.21.15.89:0/1236688649 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8c340180a0 con 0x7f8c3c0cd180 2023-07-06T02:23:43.681 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8c2006ad10 msgr2=0x7f8c2006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8c2006ad10 0x7f8c2006d1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38008040 comp rx=0 tx=0).stop 2023-07-06T02:23:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 msgr2=0x7f8c3c15b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15b130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c34002230 tx=0x7f8c3400a060 comp rx=0 tx=0).stop 2023-07-06T02:23:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 shutdown_connections 2023-07-06T02:23:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f8c2006ad10 0x7f8c2006d1c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c0cd180 0x7f8c3c15b130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c3c06dc10 0x7f8c3c148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 --2- 172.21.15.89:0/1236688649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c3c0705a0 0x7f8c3c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 >> 172.21.15.89:0/1236688649 conn(0x7f8c3c0d0b00 msgr2=0x7f8c3c070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 shutdown_connections 2023-07-06T02:23:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.681+0000 7f8c43f0b700 1 -- 172.21.15.89:0/1236688649 wait complete. 2023-07-06T02:23:43.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:43.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.816+0000 7f3eb3c5f700 1 Processor -- start 2023-07-06T02:23:43.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.816+0000 7f3eb3c5f700 1 -- start start 2023-07-06T02:23:43.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb3c5f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3eac06c5e0 0x7f3eac0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb3c5f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eac160b80 0x7f3eac0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb3c5f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac160fe0 con 0x7f3eac159fd0 2023-07-06T02:23:43.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb3c5f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac0cdb60 con 0x7f3eac160b80 2023-07-06T02:23:43.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb3c5f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac15ce60 con 0x7f3eac06c5e0 2023-07-06T02:23:43.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb21fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3eac160b80 0x7f3eac0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb19fb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3eac159fd0 0x7f3eac06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58656/0 (socket says 172.21.15.89:58656) 2023-07-06T02:23:43.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb19fb700 1 -- 172.21.15.89:0/2520466537 learned_addr learned my addr 172.21.15.89:0/2520466537 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:43.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb11fa700 1 --2- 172.21.15.89:0/2520466537 >> v2:172.21.15.172:3300/0 conn(0x7f3eac06c5e0 0x7f3eac0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb21fc700 1 -- 172.21.15.89:0/2520466537 >> v2:172.21.15.172:3300/0 conn(0x7f3eac06c5e0 msgr2=0x7f3eac0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb21fc700 1 --2- 172.21.15.89:0/2520466537 >> v2:172.21.15.172:3300/0 conn(0x7f3eac06c5e0 0x7f3eac0cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb21fc700 1 -- 172.21.15.89:0/2520466537 >> v1:172.21.15.89:6789/0 conn(0x7f3eac159fd0 legacy=0x7f3eac06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb21fc700 1 -- 172.21.15.89:0/2520466537 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac15d140 con 0x7f3eac160b80 2023-07-06T02:23:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb09f9700 1 -- 172.21.15.89:0/2520466537 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011038734 0 0) 0x7f3eac160fe0 con 0x7f3eac159fd0 2023-07-06T02:23:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb21fc700 1 --2- 172.21.15.89:0/2520466537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac160b80 0x7f3eac0d0a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0cf730 tx=0x7f3ea400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bad3b4225678d5c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb09f9700 1 -- 172.21.15.89:0/2520466537 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea400e070 con 0x7f3eac160b80 2023-07-06T02:23:43.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.817+0000 7f3eb09f9700 1 -- 172.21.15.89:0/2520466537 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea400bbd0 con 0x7f3eac160b80 2023-07-06T02:23:43.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb09f9700 1 -- 172.21.15.89:0/2520466537 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4012430 con 0x7f3eac160b80 2023-07-06T02:23:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/2520466537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac160b80 msgr2=0x7f3eac0d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/2520466537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac160b80 0x7f3eac0d0a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0cf730 tx=0x7f3ea400b330 comp rx=0 tx=0).stop 2023-07-06T02:23:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/2520466537 shutdown_connections 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/2520466537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac160b80 0x7f3eac0d0a50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/2520466537 >> v2:172.21.15.172:3300/0 conn(0x7f3eac06c5e0 0x7f3eac0cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/2520466537 >> 172.21.15.89:0/2520466537 conn(0x7f3eac0d0590 msgr2=0x7f3eac0d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/2520466537 shutdown_connections 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.818+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/2520466537 wait complete. 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.819+0000 7f3eb3c5f700 1 Processor -- start 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.819+0000 7f3eb3c5f700 1 -- start start 2023-07-06T02:23:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb3c5f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac06c5e0 0x7f3eac0b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb3c5f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eac159fd0 0x7f3eac0c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb3c5f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eac160b80 0x7f3eac0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb3c5f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0b8bf0 con 0x7f3eac06c5e0 2023-07-06T02:23:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb3c5f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1f1c70 con 0x7f3eac160b80 2023-07-06T02:23:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb3c5f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0b8630 con 0x7f3eac159fd0 2023-07-06T02:23:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac06c5e0 0x7f3eac0b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac06c5e0 0x7f3eac0b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33760/0 (socket says 172.21.15.89:33760) 2023-07-06T02:23:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 -- 172.21.15.89:0/3708365089 learned_addr learned my addr 172.21.15.89:0/3708365089 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 -- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eac159fd0 msgr2=0x7f3eac0c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb11fa700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eac159fd0 0x7f3eac0c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb21fc700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eac160b80 0x7f3eac0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eac159fd0 0x7f3eac0c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 -- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eac160b80 msgr2=0x7f3eac0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eac160b80 0x7f3eac0c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.820+0000 7f3eb19fb700 1 -- 172.21.15.89:0/3708365089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea4009a70 con 0x7f3eac06c5e0 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.821+0000 7f3eb19fb700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac06c5e0 0x7f3eac0b7c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0011e0 tx=0x7f3e9c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.821+0000 7f3ea27fc700 1 -- 172.21.15.89:0/3708365089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c004c80 con 0x7f3eac06c5e0 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.821+0000 7f3ea27fc700 1 -- 172.21.15.89:0/3708365089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c004e00 con 0x7f3eac06c5e0 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.821+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac0b8910 con 0x7f3eac06c5e0 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.821+0000 7f3ea27fc700 1 -- 172.21.15.89:0/3708365089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c007310 con 0x7f3eac06c5e0 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.821+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac142520 con 0x7f3eac06c5e0 2023-07-06T02:23:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.822+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac1420a0 con 0x7f3eac06c5e0 2023-07-06T02:23:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.822+0000 7f3ea27fc700 1 -- 172.21.15.89:0/3708365089 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e9c007470 con 0x7f3eac06c5e0 2023-07-06T02:23:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.822+0000 7f3ea27fc700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.823+0000 7f3ea27fc700 1 -- 172.21.15.89:0/3708365089 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3e9c089530 con 0x7f3eac06c5e0 2023-07-06T02:23:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.823+0000 7f3eb11fa700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.823+0000 7f3eb11fa700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9006ace0 0x7f3e9006d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800b840 tx=0x7f3ea8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:43.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:43.827+0000 7f3ea27fc700 1 -- 172.21.15.89:0/3708365089 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3eac1420a0 con 0x7f3eac06c5e0 2023-07-06T02:23:44.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.013+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3eac1420a0 con 0x7f3eac06c5e0 2023-07-06T02:23:44.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.013+0000 7f3ea27fc700 1 -- 172.21.15.89:0/3708365089 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3eac1420a0 con 0x7f3eac06c5e0 2023-07-06T02:23:44.016 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.015+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9006ace0 msgr2=0x7f3e9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.015+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9006ace0 0x7f3e9006d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800b840 tx=0x7f3ea8008040 comp rx=0 tx=0).stop 2023-07-06T02:23:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac06c5e0 msgr2=0x7f3eac0b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac06c5e0 0x7f3eac0b7c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0011e0 tx=0x7f3e9c00cae0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 shutdown_connections 2023-07-06T02:23:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eac160b80 0x7f3eac0c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3eac06c5e0 0x7f3eac0b7c20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 --2- 172.21.15.89:0/3708365089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eac159fd0 0x7f3eac0c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 >> 172.21.15.89:0/3708365089 conn(0x7f3eac0d0590 msgr2=0x7f3eac159100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 shutdown_connections 2023-07-06T02:23:44.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.016+0000 7f3eb3c5f700 1 -- 172.21.15.89:0/3708365089 wait complete. 2023-07-06T02:23:44.030 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:44.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.150+0000 7f653f8a3700 1 Processor -- start 2023-07-06T02:23:44.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.151+0000 7f653f8a3700 1 -- start start 2023-07-06T02:23:44.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.151+0000 7f653f8a3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6538071630 0x7f65380caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.151+0000 7f653f8a3700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653806a500 con 0x7f653815a3e0 2023-07-06T02:23:44.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.151+0000 7f653f8a3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653806ec60 con 0x7f653806bcd0 2023-07-06T02:23:44.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.151+0000 7f653f8a3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653806f190 con 0x7f6538071630 2023-07-06T02:23:44.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f653d63f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f653815a3e0 0x7f6538070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58670/0 (socket says 172.21.15.89:58670) 2023-07-06T02:23:44.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f653d63f700 1 -- 172.21.15.89:0/935442911 learned_addr learned my addr 172.21.15.89:0/935442911 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:44.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f653ce3e700 1 --2- 172.21.15.89:0/935442911 >> v2:172.21.15.172:3300/0 conn(0x7f6538071630 0x7f65380caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442367575 0 0) 0x7f653806a500 con 0x7f653815a3e0 2023-07-06T02:23:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65240035b0 con 0x7f653815a3e0 2023-07-06T02:23:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3831769662 0 0) 0x7f65240035b0 con 0x7f653815a3e0 2023-07-06T02:23:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 >> v2:172.21.15.172:3300/0 conn(0x7f6538071630 msgr2=0x7f65380caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 --2- 172.21.15.89:0/935442911 >> v2:172.21.15.172:3300/0 conn(0x7f6538071630 0x7f65380caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 >> v1:172.21.15.172:6789/0 conn(0x7f653806bcd0 legacy=0x7f653815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380cb980 con 0x7f653815a3e0 2023-07-06T02:23:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f6528001670 con 0x7f653815a3e0 2023-07-06T02:23:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 >> v1:172.21.15.89:6789/0 conn(0x7f653815a3e0 legacy=0x7f6538070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524003a00 0x7f6524003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524004850 0x7f6524004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f652400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653806f190 con 0x7f6524005200 2023-07-06T02:23:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652400c110 con 0x7f6524004850 2023-07-06T02:23:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652400c2d0 con 0x7f6524003a00 2023-07-06T02:23:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f653de40700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f652400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f653d63f700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524003a00 0x7f6524003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.152+0000 7f653ce3e700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524004850 0x7f6524004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653de40700 1 -- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524003a00 msgr2=0x7f6524003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653de40700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524003a00 0x7f6524003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653de40700 1 -- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524004850 msgr2=0x7f6524004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653de40700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524004850 0x7f6524004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653de40700 1 -- 172.21.15.89:0/935442911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240035b0 con 0x7f6524005200 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653de40700 1 -- 172.21.15.89:0/935442911 reap_dead start 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653de40700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f652400ba10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f653000c190 tx=0x7f6524005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaf045499d294e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530001db0 con 0x7f6524005200 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653000ccf0 con 0x7f6524005200 2023-07-06T02:23:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f652ffff700 1 -- 172.21.15.89:0/935442911 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530003c40 con 0x7f6524005200 2023-07-06T02:23:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653f8a3700 1 -- 172.21.15.89:0/935442911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 msgr2=0x7f652400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653f8a3700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f652400ba10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f653000c190 tx=0x7f6524005880 comp rx=0 tx=0).stop 2023-07-06T02:23:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653f8a3700 1 -- 172.21.15.89:0/935442911 shutdown_connections 2023-07-06T02:23:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653f8a3700 1 --2- 172.21.15.89:0/935442911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f652400ba10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653f8a3700 1 -- 172.21.15.89:0/935442911 >> 172.21.15.89:0/935442911 conn(0x7f65380cf1c0 msgr2=0x7f65380cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653f8a3700 1 -- 172.21.15.89:0/935442911 shutdown_connections 2023-07-06T02:23:44.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.153+0000 7f653f8a3700 1 -- 172.21.15.89:0/935442911 wait complete. 2023-07-06T02:23:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 Processor -- start 2023-07-06T02:23:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 -- start start 2023-07-06T02:23:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f65380cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380c4980 0x7f6538157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65380b36b0 0x7f65380b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380b4060 con 0x7f6524005200 2023-07-06T02:23:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380cde30 con 0x7f65380c4980 2023-07-06T02:23:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653f8a3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65380ce110 con 0x7f65380b36b0 2023-07-06T02:23:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653d63f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f65380cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653d63f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f65380cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33780/0 (socket says 172.21.15.89:33780) 2023-07-06T02:23:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.155+0000 7f653d63f700 1 -- 172.21.15.89:0/3971891804 learned_addr learned my addr 172.21.15.89:0/3971891804 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653ce3e700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380c4980 0x7f6538157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653de40700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65380b36b0 0x7f65380b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653d63f700 1 -- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65380b36b0 msgr2=0x7f65380b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653d63f700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65380b36b0 0x7f65380b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653d63f700 1 -- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380c4980 msgr2=0x7f6538157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653d63f700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380c4980 0x7f6538157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653d63f700 1 -- 172.21.15.89:0/3971891804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380ce3f0 con 0x7f6524005200 2023-07-06T02:23:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653d63f700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f65380cd7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6528007290 tx=0x7f65280075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f652dffb700 1 -- 172.21.15.89:0/3971891804 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800f660 con 0x7f6524005200 2023-07-06T02:23:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f652dffb700 1 -- 172.21.15.89:0/3971891804 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652800f7c0 con 0x7f6524005200 2023-07-06T02:23:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65380ce6d0 con 0x7f6524005200 2023-07-06T02:23:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f652dffb700 1 -- 172.21.15.89:0/3971891804 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800fb70 con 0x7f6524005200 2023-07-06T02:23:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.156+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65380c4df0 con 0x7f6524005200 2023-07-06T02:23:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.157+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65380ce250 con 0x7f6524005200 2023-07-06T02:23:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.157+0000 7f652dffb700 1 -- 172.21.15.89:0/3971891804 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f652800fcd0 con 0x7f6524005200 2023-07-06T02:23:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.158+0000 7f652dffb700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f651c06ace0 0x7f651c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.158+0000 7f652dffb700 1 -- 172.21.15.89:0/3971891804 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f65280897b0 con 0x7f6524005200 2023-07-06T02:23:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.158+0000 7f653ce3e700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f651c06ace0 0x7f651c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.158+0000 7f653ce3e700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f651c06ace0 0x7f651c06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65380ae490 tx=0x7f6534008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.162+0000 7f652dffb700 1 -- 172.21.15.89:0/3971891804 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65380ce250 con 0x7f6524005200 2023-07-06T02:23:44.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.350+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65380cebf0 con 0x7f6524005200 2023-07-06T02:23:44.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.350+0000 7f652dffb700 1 -- 172.21.15.89:0/3971891804 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65380cebf0 con 0x7f6524005200 2023-07-06T02:23:44.352 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f651c06ace0 msgr2=0x7f651c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f651c06ace0 0x7f651c06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65380ae490 tx=0x7f6534008040 comp rx=0 tx=0).stop 2023-07-06T02:23:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 msgr2=0x7f65380cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f65380cd7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6528007290 tx=0x7f65280075a0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 shutdown_connections 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65380b36b0 0x7f65380b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f651c06ace0 0x7f651c06d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6524005200 0x7f65380cd7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 --2- 172.21.15.89:0/3971891804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380c4980 0x7f6538157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 >> 172.21.15.89:0/3971891804 conn(0x7f65380cf1c0 msgr2=0x7f6538159690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 shutdown_connections 2023-07-06T02:23:44.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.352+0000 7f653f8a3700 1 -- 172.21.15.89:0/3971891804 wait complete. 2023-07-06T02:23:44.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:44.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.489+0000 7ff7d257f700 1 Processor -- start 2023-07-06T02:23:44.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.489+0000 7ff7d257f700 1 -- start start 2023-07-06T02:23:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.489+0000 7ff7d257f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0b5e50 0x7ff7cc070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.489+0000 7ff7d257f700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc0cb150 con 0x7ff7cc070870 2023-07-06T02:23:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.489+0000 7ff7d257f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc0cb430 con 0x7ff7cc15c610 2023-07-06T02:23:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.489+0000 7ff7d257f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7cc0d05e0 con 0x7ff7cc0b5e50 2023-07-06T02:23:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7d0b1c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff7cc070870 0x7ff7cc0cb8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58694/0 (socket says 172.21.15.89:58694) 2023-07-06T02:23:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7d0b1c700 1 -- 172.21.15.89:0/1046796970 learned_addr learned my addr 172.21.15.89:0/1046796970 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7cb7fe700 1 --2- 172.21.15.89:0/1046796970 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0b5e50 0x7ff7cc070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972357387 0 0) 0x7ff7cc0cb150 con 0x7ff7cc070870 2023-07-06T02:23:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b80035b0 con 0x7ff7cc070870 2023-07-06T02:23:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1702819943 0 0) 0x7ff7b80035b0 con 0x7ff7cc070870 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0b5e50 msgr2=0x7ff7cc070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 --2- 172.21.15.89:0/1046796970 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0b5e50 0x7ff7cc070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 >> v1:172.21.15.172:6789/0 conn(0x7ff7cc15c610 legacy=0x7ff7cc0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc070c60 con 0x7ff7cc070870 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7ff7c40019f0 con 0x7ff7cc070870 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 >> v1:172.21.15.89:6789/0 conn(0x7ff7cc070870 legacy=0x7ff7cc0cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b8003a00 0x7ff7b8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b8004850 0x7ff7b8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0d05e0 con 0x7ff7b8005200 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b800c080 con 0x7ff7b8004850 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b800c240 con 0x7ff7b8003a00 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7d0b1c700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7cb7fe700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b8004850 0x7ff7b8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.490+0000 7ff7cbfff700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b8003a00 0x7ff7b8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d0b1c700 1 -- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b8003a00 msgr2=0x7ff7b8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d0b1c700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b8003a00 0x7ff7b8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d0b1c700 1 -- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b8004850 msgr2=0x7ff7b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d0b1c700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b8004850 0x7ff7b8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d0b1c700 1 -- 172.21.15.89:0/1046796970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80035b0 con 0x7ff7b8005200 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d0b1c700 1 -- 172.21.15.89:0/1046796970 reap_dead start 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d0b1c700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7b800ba10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400a820 tx=0x7ff7b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d758e89cbed8a6a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400a2f0 con 0x7ff7b8005200 2023-07-06T02:23:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4012620 con 0x7ff7b8005200 2023-07-06T02:23:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7caffd700 1 -- 172.21.15.89:0/1046796970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40129d0 con 0x7ff7b8005200 2023-07-06T02:23:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d257f700 1 -- 172.21.15.89:0/1046796970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 msgr2=0x7ff7b800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d257f700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7b800ba10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400a820 tx=0x7ff7b8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d257f700 1 -- 172.21.15.89:0/1046796970 shutdown_connections 2023-07-06T02:23:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d257f700 1 --2- 172.21.15.89:0/1046796970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7b800ba10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d257f700 1 -- 172.21.15.89:0/1046796970 >> 172.21.15.89:0/1046796970 conn(0x7ff7cc0d0b00 msgr2=0x7ff7cc0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.491+0000 7ff7d257f700 1 -- 172.21.15.89:0/1046796970 shutdown_connections 2023-07-06T02:23:44.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.492+0000 7ff7d257f700 1 -- 172.21.15.89:0/1046796970 wait complete. 2023-07-06T02:23:44.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.493+0000 7ff7d257f700 1 Processor -- start 2023-07-06T02:23:44.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.493+0000 7ff7d257f700 1 -- start start 2023-07-06T02:23:44.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7d257f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7cc15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7d257f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc1f2c10 0x7ff7cc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7d257f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc1f4b60 0x7ff7cc1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7d257f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc15da60 con 0x7ff7b8005200 2023-07-06T02:23:44.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7d257f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc15d4a0 con 0x7ff7cc1f2c10 2023-07-06T02:23:44.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7d257f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc15d780 con 0x7ff7cc1f4b60 2023-07-06T02:23:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7cc15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7cc15c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33800/0 (socket says 172.21.15.89:33800) 2023-07-06T02:23:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7cbfff700 1 -- 172.21.15.89:0/2649095693 learned_addr learned my addr 172.21.15.89:0/2649095693 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7d0b1c700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc1f4b60 0x7ff7cc1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.494+0000 7ff7cb7fe700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc1f2c10 0x7ff7cc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7cbfff700 1 -- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc1f4b60 msgr2=0x7ff7cc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7cbfff700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc1f4b60 0x7ff7cc1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7cbfff700 1 -- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc1f2c10 msgr2=0x7ff7cc155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7cbfff700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc1f2c10 0x7ff7cc155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7cbfff700 1 -- 172.21.15.89:0/2649095693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc15dfb0 con 0x7ff7b8005200 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7d0b1c700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc1f4b60 0x7ff7cc1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7cbfff700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7cc15c9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00f820 tx=0x7ff7bc00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7c8ff9700 1 -- 172.21.15.89:0/2649095693 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0052e0 con 0x7ff7b8005200 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7c8ff9700 1 -- 172.21.15.89:0/2649095693 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc005460 con 0x7ff7b8005200 2023-07-06T02:23:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7c8ff9700 1 -- 172.21.15.89:0/2649095693 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0038c0 con 0x7ff7b8005200 2023-07-06T02:23:44.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc15e290 con 0x7ff7b8005200 2023-07-06T02:23:44.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.495+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc15cf20 con 0x7ff7b8005200 2023-07-06T02:23:44.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.496+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc15de10 con 0x7ff7b8005200 2023-07-06T02:23:44.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.496+0000 7ff7c8ff9700 1 -- 172.21.15.89:0/2649095693 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7bc003a20 con 0x7ff7b8005200 2023-07-06T02:23:44.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.496+0000 7ff7c8ff9700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7ac06ace0 0x7ff7ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.497+0000 7ff7c8ff9700 1 -- 172.21.15.89:0/2649095693 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff7bc089550 con 0x7ff7b8005200 2023-07-06T02:23:44.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.497+0000 7ff7cb7fe700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7ac06ace0 0x7ff7ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.497+0000 7ff7cb7fe700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7ac06ace0 0x7ff7ac06d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009b20 tx=0x7ff7b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.501+0000 7ff7c8ff9700 1 -- 172.21.15.89:0/2649095693 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff7cc15de10 con 0x7ff7b8005200 2023-07-06T02:23:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.687+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7cc15de10 con 0x7ff7b8005200 2023-07-06T02:23:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.687+0000 7ff7c8ff9700 1 -- 172.21.15.89:0/2649095693 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7cc15de10 con 0x7ff7b8005200 2023-07-06T02:23:44.689 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7ac06ace0 msgr2=0x7ff7ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7ac06ace0 0x7ff7ac06d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009b20 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2023-07-06T02:23:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 msgr2=0x7ff7cc15c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7cc15c9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00f820 tx=0x7ff7bc00fbe0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 shutdown_connections 2023-07-06T02:23:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc1f4b60 0x7ff7cc1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff7ac06ace0 0x7ff7ac06d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b8005200 0x7ff7cc15c9e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 --2- 172.21.15.89:0/2649095693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc1f2c10 0x7ff7cc155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 >> 172.21.15.89:0/2649095693 conn(0x7ff7cc0d0b00 msgr2=0x7ff7cc0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 shutdown_connections 2023-07-06T02:23:44.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.689+0000 7ff7d257f700 1 -- 172.21.15.89:0/2649095693 wait complete. 2023-07-06T02:23:44.703 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:44.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.824+0000 7f492e61a700 1 Processor -- start 2023-07-06T02:23:44.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.824+0000 7f492e61a700 1 -- start start 2023-07-06T02:23:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.824+0000 7f492e61a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f492815a3e0 0x7f49280caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.824+0000 7f492e61a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4928071630 0x7f492815dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.824+0000 7f492e61a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492806a500 con 0x7f492806bcd0 2023-07-06T02:23:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.824+0000 7f492e61a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492806ec60 con 0x7f492815a3e0 2023-07-06T02:23:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.824+0000 7f492e61a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492806f190 con 0x7f4928071630 2023-07-06T02:23:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4927fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f492815a3e0 0x7f49280caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f49277fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f492806bcd0 0x7f4928070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58710/0 (socket says 172.21.15.89:58710) 2023-07-06T02:23:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4927fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f492815a3e0 0x7f49280caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33810/0 (socket says 172.21.15.89:33810) 2023-07-06T02:23:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f49277fe700 1 -- 172.21.15.89:0/1240480151 learned_addr learned my addr 172.21.15.89:0/1240480151 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4927fff700 1 -- 172.21.15.89:0/1240480151 >> v2:172.21.15.172:3300/0 conn(0x7f4928071630 msgr2=0x7f492815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f492cbb7700 1 --2- 172.21.15.89:0/1240480151 >> v2:172.21.15.172:3300/0 conn(0x7f4928071630 0x7f492815dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4927fff700 1 --2- 172.21.15.89:0/1240480151 >> v2:172.21.15.172:3300/0 conn(0x7f4928071630 0x7f492815dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4926ffd700 1 -- 172.21.15.89:0/1240480151 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633110670 0 0) 0x7f492806a500 con 0x7f492806bcd0 2023-07-06T02:23:44.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4927fff700 1 -- 172.21.15.89:0/1240480151 >> v1:172.21.15.89:6789/0 conn(0x7f492806bcd0 legacy=0x7f4928070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4927fff700 1 -- 172.21.15.89:0/1240480151 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280cb980 con 0x7f492815a3e0 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4927fff700 1 --2- 172.21.15.89:0/1240480151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280caf80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49280d0270 tx=0x7f4918009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c703b4c3ec95e949 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4926ffd700 1 -- 172.21.15.89:0/1240480151 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918011830 con 0x7f492815a3e0 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4926ffd700 1 -- 172.21.15.89:0/1240480151 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4918011990 con 0x7f492815a3e0 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.825+0000 7f4926ffd700 1 -- 172.21.15.89:0/1240480151 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918011d20 con 0x7f492815a3e0 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 -- 172.21.15.89:0/1240480151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 msgr2=0x7f49280caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 --2- 172.21.15.89:0/1240480151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280caf80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49280d0270 tx=0x7f4918009b20 comp rx=0 tx=0).stop 2023-07-06T02:23:44.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 -- 172.21.15.89:0/1240480151 shutdown_connections 2023-07-06T02:23:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 --2- 172.21.15.89:0/1240480151 >> v2:172.21.15.172:3300/0 conn(0x7f4928071630 0x7f492815dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 --2- 172.21.15.89:0/1240480151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280caf80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 -- 172.21.15.89:0/1240480151 >> 172.21.15.89:0/1240480151 conn(0x7f49280cf1c0 msgr2=0x7f49280cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 -- 172.21.15.89:0/1240480151 shutdown_connections 2023-07-06T02:23:44.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.826+0000 7f492e61a700 1 -- 172.21.15.89:0/1240480151 wait complete. 2023-07-06T02:23:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.827+0000 7f492e61a700 1 Processor -- start 2023-07-06T02:23:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.827+0000 7f492e61a700 1 -- start start 2023-07-06T02:23:44.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492e61a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492806bcd0 0x7f49280b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492e61a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4928071630 0x7f49280ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492e61a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492e61a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49280c84c0 con 0x7f492815a3e0 2023-07-06T02:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492e61a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49280c8740 con 0x7f492806bcd0 2023-07-06T02:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492e61a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49280c8a20 con 0x7f4928071630 2023-07-06T02:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33818/0 (socket says 172.21.15.89:33818) 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 -- 172.21.15.89:0/3833748269 learned_addr learned my addr 172.21.15.89:0/3833748269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f4927fff700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492806bcd0 0x7f49280b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 -- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4928071630 msgr2=0x7f49280ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f49277fe700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4928071630 0x7f49280ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4928071630 0x7f49280ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 -- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492806bcd0 msgr2=0x7f49280b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492806bcd0 0x7f49280b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:44.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 -- 172.21.15.89:0/3833748269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180094a0 con 0x7f492815a3e0 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f4927fff700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492806bcd0 0x7f49280b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f49277fe700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4928071630 0x7f49280ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.828+0000 7f492cbb7700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280cd2d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f492000cc80 tx=0x7f492000a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.829+0000 7f4924ff9700 1 -- 172.21.15.89:0/3833748269 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492000ce20 con 0x7f492815a3e0 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.829+0000 7f4924ff9700 1 -- 172.21.15.89:0/3833748269 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492000ad30 con 0x7f492815a3e0 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.829+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49280c8d00 con 0x7f492815a3e0 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.829+0000 7f4924ff9700 1 -- 172.21.15.89:0/3833748269 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492000aea0 con 0x7f492815a3e0 2023-07-06T02:23:44.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.829+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4928152ae0 con 0x7f492815a3e0 2023-07-06T02:23:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.830+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49280c9220 con 0x7f492815a3e0 2023-07-06T02:23:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.830+0000 7f4924ff9700 1 -- 172.21.15.89:0/3833748269 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f492000f3f0 con 0x7f492815a3e0 2023-07-06T02:23:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.830+0000 7f4924ff9700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f491406ace0 0x7f491406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.831+0000 7f4924ff9700 1 -- 172.21.15.89:0/3833748269 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f49200899d0 con 0x7f492815a3e0 2023-07-06T02:23:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.831+0000 7f4927fff700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f491406ace0 0x7f491406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:44.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.831+0000 7f4927fff700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f491406ace0 0x7f491406d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49180090b0 tx=0x7f4918008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:44.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:44.835+0000 7f4924ff9700 1 -- 172.21.15.89:0/3833748269 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4920054750 con 0x7f492815a3e0 2023-07-06T02:23:45.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.023+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4928074270 con 0x7f492815a3e0 2023-07-06T02:23:45.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.023+0000 7f4924ff9700 1 -- 172.21.15.89:0/3833748269 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f492001d0a0 con 0x7f492815a3e0 2023-07-06T02:23:45.026 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:45.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f491406ace0 msgr2=0x7f491406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f491406ace0 0x7f491406d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49180090b0 tx=0x7f4918008030 comp rx=0 tx=0).stop 2023-07-06T02:23:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 msgr2=0x7f49280cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280cd2d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f492000cc80 tx=0x7f492000a4d0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 shutdown_connections 2023-07-06T02:23:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f491406ace0 0x7f491406d190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f492815a3e0 0x7f49280cd2d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492806bcd0 0x7f49280b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 --2- 172.21.15.89:0/3833748269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4928071630 0x7f49280ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.025+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 >> 172.21.15.89:0/3833748269 conn(0x7f49280cf1c0 msgr2=0x7f49280cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.026+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 shutdown_connections 2023-07-06T02:23:45.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.026+0000 7f492e61a700 1 -- 172.21.15.89:0/3833748269 wait complete. 2023-07-06T02:23:45.039 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T02:23:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.158+0000 7f37c08e6700 1 Processor -- start 2023-07-06T02:23:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.158+0000 7f37c08e6700 1 -- start start 2023-07-06T02:23:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.159+0000 7f37c08e6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37bc0705a0 0x7f37bc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.159+0000 7f37c08e6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc06e110 con 0x7f37bc0cd180 2023-07-06T02:23:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.159+0000 7f37c08e6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc06e3f0 con 0x7f37bc06dc10 2023-07-06T02:23:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.159+0000 7f37c08e6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc06e6d0 con 0x7f37bc0705a0 2023-07-06T02:23:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37ba59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37bc0705a0 0x7f37bc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37b9d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f37bc0cd180 0x7f37bc15a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58724/0 (socket says 172.21.15.89:58724) 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37ba59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37bc0705a0 0x7f37bc0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33822/0 (socket says 172.21.15.89:33822) 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37b9d9b700 1 -- 172.21.15.89:0/952144403 learned_addr learned my addr 172.21.15.89:0/952144403 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37ba59c700 1 -- 172.21.15.89:0/952144403 >> v1:172.21.15.172:6789/0 conn(0x7f37bc06dc10 legacy=0x7f37bc157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37ba59c700 1 -- 172.21.15.89:0/952144403 >> v1:172.21.15.89:6789/0 conn(0x7f37bc0cd180 legacy=0x7f37bc15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37b959a700 1 -- 172.21.15.89:0/952144403 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182876862 0 0) 0x7f37bc06e110 con 0x7f37bc0cd180 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37ba59c700 1 -- 172.21.15.89:0/952144403 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc06e9b0 con 0x7f37bc0705a0 2023-07-06T02:23:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37ba59c700 1 --2- 172.21.15.89:0/952144403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc0705a0 0x7f37bc0ccc40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009840 tx=0x7f37ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8998aa52654bcc08 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37b959a700 1 -- 172.21.15.89:0/952144403 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac004230 con 0x7f37bc0705a0 2023-07-06T02:23:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37b959a700 1 -- 172.21.15.89:0/952144403 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac004390 con 0x7f37bc0705a0 2023-07-06T02:23:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.160+0000 7f37b959a700 1 -- 172.21.15.89:0/952144403 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0036a0 con 0x7f37bc0705a0 2023-07-06T02:23:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.161+0000 7f37c08e6700 1 -- 172.21.15.89:0/952144403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc0705a0 msgr2=0x7f37bc0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.161+0000 7f37c08e6700 1 --2- 172.21.15.89:0/952144403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc0705a0 0x7f37bc0ccc40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009840 tx=0x7f37ac009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.161+0000 7f37c08e6700 1 -- 172.21.15.89:0/952144403 shutdown_connections 2023-07-06T02:23:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.161+0000 7f37c08e6700 1 --2- 172.21.15.89:0/952144403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc0705a0 0x7f37bc0ccc40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.161+0000 7f37c08e6700 1 -- 172.21.15.89:0/952144403 >> 172.21.15.89:0/952144403 conn(0x7f37bc0d0b00 msgr2=0x7f37bc0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.161+0000 7f37c08e6700 1 -- 172.21.15.89:0/952144403 shutdown_connections 2023-07-06T02:23:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.161+0000 7f37c08e6700 1 -- 172.21.15.89:0/952144403 wait complete. 2023-07-06T02:23:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.162+0000 7f37c08e6700 1 Processor -- start 2023-07-06T02:23:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.163+0000 7f37c08e6700 1 -- start start 2023-07-06T02:23:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.163+0000 7f37c08e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc06dc10 0x7f37bc0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.163+0000 7f37c08e6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc0705a0 0x7f37bc0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.163+0000 7f37c08e6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37bc0cd180 0x7f37bc0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.163+0000 7f37c08e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0b6660 con 0x7f37bc06dc10 2023-07-06T02:23:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc06dc10 0x7f37bc0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37c08e6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0b5080 con 0x7f37bc0705a0 2023-07-06T02:23:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37c08e6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0b5360 con 0x7f37bc0cd180 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc06dc10 0x7f37bc0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33828/0 (socket says 172.21.15.89:33828) 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 -- 172.21.15.89:0/2396109227 learned_addr learned my addr 172.21.15.89:0/2396109227 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37b9d9b700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc0705a0 0x7f37bc0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37bad9d700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37bc0cd180 0x7f37bc0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 -- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37bc0cd180 msgr2=0x7f37bc0b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37bc0cd180 0x7f37bc0b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 -- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc0705a0 msgr2=0x7f37bc0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc0705a0 0x7f37bc0b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 -- 172.21.15.89:0/2396109227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0094a0 con 0x7f37bc06dc10 2023-07-06T02:23:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37bad9d700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37bc0cd180 0x7f37bc0b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37ba59c700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc06dc10 0x7f37bc0c7570 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0d18e0 tx=0x7f37ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37b2ffd700 1 -- 172.21.15.89:0/2396109227 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac011630 con 0x7f37bc06dc10 2023-07-06T02:23:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37b2ffd700 1 -- 172.21.15.89:0/2396109227 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac01e070 con 0x7f37bc06dc10 2023-07-06T02:23:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc0b5640 con 0x7f37bc06dc10 2023-07-06T02:23:45.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37b2ffd700 1 -- 172.21.15.89:0/2396109227 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0119b0 con 0x7f37bc06dc10 2023-07-06T02:23:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.164+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc0b5c00 con 0x7f37bc06dc10 2023-07-06T02:23:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.165+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc074270 con 0x7f37bc06dc10 2023-07-06T02:23:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.165+0000 7f37b2ffd700 1 -- 172.21.15.89:0/2396109227 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f37ac011b10 con 0x7f37bc06dc10 2023-07-06T02:23:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.166+0000 7f37b2ffd700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f379c06ace0 0x7f379c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.166+0000 7f37b2ffd700 1 -- 172.21.15.89:0/2396109227 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f37ac08ab10 con 0x7f37bc06dc10 2023-07-06T02:23:45.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.166+0000 7f37b9d9b700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f379c06ace0 0x7f379c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.166+0000 7f37b9d9b700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f379c06ace0 0x7f379c06d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37bc1558d0 tx=0x7f37a4007d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:45.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.170+0000 7f37b2ffd700 1 -- 172.21.15.89:0/2396109227 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f37ac055920 con 0x7f37bc06dc10 2023-07-06T02:23:45.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.374+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37bc06f590 con 0x7f37bc06dc10 2023-07-06T02:23:45.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.374+0000 7f37b2ffd700 1 -- 172.21.15.89:0/2396109227 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f37ac058f40 con 0x7f37bc06dc10 2023-07-06T02:23:45.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f379c06ace0 msgr2=0x7f379c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f379c06ace0 0x7f379c06d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37bc1558d0 tx=0x7f37a4007d90 comp rx=0 tx=0).stop 2023-07-06T02:23:45.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc06dc10 msgr2=0x7f37bc0c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc06dc10 0x7f37bc0c7570 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0d18e0 tx=0x7f37ac004750 comp rx=0 tx=0).stop 2023-07-06T02:23:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 shutdown_connections 2023-07-06T02:23:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37bc0cd180 0x7f37bc0b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f379c06ace0 0x7f379c06d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc06dc10 0x7f37bc0c7570 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 --2- 172.21.15.89:0/2396109227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc0705a0 0x7f37bc0b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 >> 172.21.15.89:0/2396109227 conn(0x7f37bc0d0b00 msgr2=0x7f37bc15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:45.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 shutdown_connections 2023-07-06T02:23:45.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.376+0000 7f37c08e6700 1 -- 172.21.15.89:0/2396109227 wait complete. 2023-07-06T02:23:45.380 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-07-06T02:23:45.390 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-06T02:23:45.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:45.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-07-06T02:23:45.446 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:23:45.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:45.447 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T02:23:45.465 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:45.465 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:45.485 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:45.485 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:45.543 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:45.544 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:45.602 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:45.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.722+0000 7f7f1312d700 1 Processor -- start 2023-07-06T02:23:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.722+0000 7f7f1312d700 1 -- start start 2023-07-06T02:23:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.722+0000 7f7f1312d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 0x7f7f0c167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.722+0000 7f7f1312d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c0cb150 con 0x7f7f0c0b5e50 2023-07-06T02:23:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.722+0000 7f7f1312d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c0cb430 con 0x7f7f0c15c610 2023-07-06T02:23:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.722+0000 7f7f1312d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0d05e0 con 0x7f7f0c070a40 2023-07-06T02:23:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7f0c0b5e50 0x7f7f0c070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58740/0 (socket says 172.21.15.89:58740) 2023-07-06T02:23:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0bfff700 1 -- 172.21.15.89:0/2590334058 learned_addr learned my addr 172.21.15.89:0/2590334058 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f116ca700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 0x7f7f0c167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795920492 0 0) 0x7f7f0c0cb150 con 0x7f7f0c0b5e50 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7efc0035b0 con 0x7f7f0c0b5e50 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3821492509 0 0) 0x7f7efc0035b0 con 0x7f7f0c0b5e50 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 msgr2=0x7f7f0c167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 0x7f7f0c167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 >> v1:172.21.15.172:6789/0 conn(0x7f7f0c15c610 legacy=0x7f7f0c0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c0b6310 con 0x7f7f0c0b5e50 2023-07-06T02:23:45.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f7ef8001670 con 0x7f7f0c0b5e50 2023-07-06T02:23:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 >> v1:172.21.15.89:6789/0 conn(0x7f7f0c0b5e50 legacy=0x7f7f0c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7efc003a30 0x7f7efc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7efc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 0x7f7efc0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0d05e0 con 0x7f7efc0043d0 2023-07-06T02:23:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc00ba10 con 0x7f7efc003a30 2023-07-06T02:23:45.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc00bba0 con 0x7f7f0c070a40 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f0bfff700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7efc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f10ec9700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7efc003a30 0x7f7efc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.723+0000 7f7f116ca700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 0x7f7efc0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0bfff700 1 -- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 msgr2=0x7f7efc0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0bfff700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 0x7f7efc0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0bfff700 1 -- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7efc003a30 msgr2=0x7f7efc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0bfff700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7efc003a30 0x7f7efc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0bfff700 1 -- 172.21.15.89:0/2590334058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0035b0 con 0x7f7efc0043d0 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f116ca700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c070a40 0x7f7efc0050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0bfff700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7efc004890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800c210 tx=0x7f7efc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05cd634bc762ee4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800cd50 con 0x7f7efc0043d0 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef8005960 con 0x7f7efc0043d0 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f0b7fe700 1 -- 172.21.15.89:0/2590334058 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8005d10 con 0x7f7efc0043d0 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f1312d700 1 -- 172.21.15.89:0/2590334058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 msgr2=0x7f7efc004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f1312d700 1 --2- 172.21.15.89:0/2590334058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7efc004890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800c210 tx=0x7f7efc005880 comp rx=0 tx=0).stop 2023-07-06T02:23:45.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f116ca700 1 -- 172.21.15.89:0/2590334058 reap_dead start 2023-07-06T02:23:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f1312d700 1 -- 172.21.15.89:0/2590334058 shutdown_connections 2023-07-06T02:23:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f1312d700 1 -- 172.21.15.89:0/2590334058 >> 172.21.15.89:0/2590334058 conn(0x7f7f0c0d0b00 msgr2=0x7f7f0c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f1312d700 1 -- 172.21.15.89:0/2590334058 shutdown_connections 2023-07-06T02:23:45.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.724+0000 7f7f1312d700 1 -- 172.21.15.89:0/2590334058 wait complete. 2023-07-06T02:23:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.726+0000 7f7f1312d700 1 Processor -- start 2023-07-06T02:23:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.726+0000 7f7f1312d700 1 -- start start 2023-07-06T02:23:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f1312d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7f0c14f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f1312d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c1f1f60 0x7f7f0c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f1312d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f0c0c8a40 0x7f7f0c152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f1312d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0c8070 con 0x7f7efc0043d0 2023-07-06T02:23:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f1312d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1f2f00 con 0x7f7f0c0c8a40 2023-07-06T02:23:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f1312d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1f2940 con 0x7f7f0c1f1f60 2023-07-06T02:23:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f10ec9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7f0c14f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f10ec9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7f0c14f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33848/0 (socket says 172.21.15.89:33848) 2023-07-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f10ec9700 1 -- 172.21.15.89:0/426534563 learned_addr learned my addr 172.21.15.89:0/426534563 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f0bfff700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c1f1f60 0x7f7f0c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.727+0000 7f7f116ca700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f0c0c8a40 0x7f7f0c152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f10ec9700 1 -- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c1f1f60 msgr2=0x7f7f0c0c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f10ec9700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c1f1f60 0x7f7f0c0c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f10ec9700 1 -- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f0c0c8a40 msgr2=0x7f7f0c152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f10ec9700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f0c0c8a40 0x7f7f0c152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f10ec9700 1 -- 172.21.15.89:0/426534563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c1f2c20 con 0x7f7efc0043d0 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f10ec9700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7f0c14f1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000f840 tx=0x7f7f0000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f097fa700 1 -- 172.21.15.89:0/426534563 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000052e0 con 0x7f7efc0043d0 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f097fa700 1 -- 172.21.15.89:0/426534563 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f00005440 con 0x7f7efc0043d0 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f097fa700 1 -- 172.21.15.89:0/426534563 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000038c0 con 0x7f7efc0043d0 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c0c7d00 con 0x7f7efc0043d0 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.728+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c0c7000 con 0x7f7efc0043d0 2023-07-06T02:23:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.729+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c0c7220 con 0x7f7efc0043d0 2023-07-06T02:23:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.729+0000 7f7f097fa700 1 -- 172.21.15.89:0/426534563 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f00003a20 con 0x7f7efc0043d0 2023-07-06T02:23:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.730+0000 7f7f097fa700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ef006ace0 0x7f7ef006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.730+0000 7f7f097fa700 1 -- 172.21.15.89:0/426534563 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7f00089600 con 0x7f7efc0043d0 2023-07-06T02:23:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.730+0000 7f7f0bfff700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ef006ace0 0x7f7ef006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.730+0000 7f7f0bfff700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ef006ace0 0x7f7ef006d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0b4ac0 tx=0x7f7ef800a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.734+0000 7f7f097fa700 1 -- 172.21.15.89:0/426534563 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f0c0c7220 con 0x7f7efc0043d0 2023-07-06T02:23:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.922+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f0c0c7220 con 0x7f7efc0043d0 2023-07-06T02:23:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.923+0000 7f7f097fa700 1 -- 172.21.15.89:0/426534563 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f7f0c0c7220 con 0x7f7efc0043d0 2023-07-06T02:23:45.926 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:45.926 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:23:41.389931+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.925+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ef006ace0 msgr2=0x7f7ef006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.925+0000 7f7f1312d700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ef006ace0 0x7f7ef006d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0b4ac0 tx=0x7f7ef800a000 comp rx=0 tx=0).stop 2023-07-06T02:23:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.925+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 msgr2=0x7f7f0c14f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.925+0000 7f7f1312d700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7f0c14f1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000f840 tx=0x7f7f0000fc00 comp rx=0 tx=0).stop 2023-07-06T02:23:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 shutdown_connections 2023-07-06T02:23:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f0c0c8a40 0x7f7f0c152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ef006ace0 0x7f7ef006d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7efc0043d0 0x7f7f0c14f1a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 --2- 172.21.15.89:0/426534563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f0c1f1f60 0x7f7f0c0c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 >> 172.21.15.89:0/426534563 conn(0x7f7f0c0d0b00 msgr2=0x7f7f0c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 shutdown_connections 2023-07-06T02:23:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:45.926+0000 7f7f1312d700 1 -- 172.21.15.89:0/426534563 wait complete. 2023-07-06T02:23:45.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:45.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:46.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.062+0000 7fccfdf06700 1 Processor -- start 2023-07-06T02:23:46.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.062+0000 7fccfdf06700 1 -- start start 2023-07-06T02:23:46.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccfdf06700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fccf81600b0 0x7fccf80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccfdf06700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccfdf06700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf80cb3d0 con 0x7fccf8156ab0 2023-07-06T02:23:46.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccfdf06700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf80b1e10 con 0x7fccf81600b0 2023-07-06T02:23:46.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccfdf06700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8156e80 con 0x7fccf80b19d0 2023-07-06T02:23:46.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fccf81600b0 0x7fccf80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf6ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fccf81600b0 0x7fccf80b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33856/0 (socket says 172.21.15.89:33856) 2023-07-06T02:23:46.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf6ffd700 1 -- 172.21.15.89:0/2232610749 learned_addr learned my addr 172.21.15.89:0/2232610749 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:46.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf7fff700 1 --2- 172.21.15.89:0/2232610749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf6ffd700 1 -- 172.21.15.89:0/2232610749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 msgr2=0x7fccf80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:46.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf6ffd700 1 --2- 172.21.15.89:0/2232610749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf6ffd700 1 -- 172.21.15.89:0/2232610749 >> v1:172.21.15.172:6789/0 conn(0x7fccf8156ab0 legacy=0x7fccf8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf6ffd700 1 -- 172.21.15.89:0/2232610749 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf80ca370 con 0x7fccf81600b0 2023-07-06T02:23:46.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.063+0000 7fccf7fff700 1 --2- 172.21.15.89:0/2232610749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:46.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccf6ffd700 1 --2- 172.21.15.89:0/2232610749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80b1490 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fccec0098f0 tx=0x7fccec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167b70e6aa75307d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccf67fc700 1 -- 172.21.15.89:0/2232610749 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00b8a0 con 0x7fccf81600b0 2023-07-06T02:23:46.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccf67fc700 1 -- 172.21.15.89:0/2232610749 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec00ba00 con 0x7fccf81600b0 2023-07-06T02:23:46.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccf67fc700 1 -- 172.21.15.89:0/2232610749 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00bd90 con 0x7fccf81600b0 2023-07-06T02:23:46.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 -- 172.21.15.89:0/2232610749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 msgr2=0x7fccf80b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2232610749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80b1490 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fccec0098f0 tx=0x7fccec009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:46.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 -- 172.21.15.89:0/2232610749 shutdown_connections 2023-07-06T02:23:46.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2232610749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2232610749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80b1490 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 -- 172.21.15.89:0/2232610749 >> 172.21.15.89:0/2232610749 conn(0x7fccf80cc490 msgr2=0x7fccf80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:46.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 -- 172.21.15.89:0/2232610749 shutdown_connections 2023-07-06T02:23:46.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.064+0000 7fccfdf06700 1 -- 172.21.15.89:0/2232610749 wait complete. 2023-07-06T02:23:46.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.065+0000 7fccfdf06700 1 Processor -- start 2023-07-06T02:23:46.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccfdf06700 1 -- start start 2023-07-06T02:23:46.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccfdf06700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccfdf06700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf8156ab0 0x7fccf814f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccfdf06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccfdf06700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf806be20 con 0x7fccf81600b0 2023-07-06T02:23:46.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccfdf06700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf806c100 con 0x7fccf8156ab0 2023-07-06T02:23:46.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccfdf06700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf806c3e0 con 0x7fccf80b19d0 2023-07-06T02:23:46.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33866/0 (socket says 172.21.15.89:33866) 2023-07-06T02:23:46.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.066+0000 7fccf7fff700 1 -- 172.21.15.89:0/2244125154 learned_addr learned my addr 172.21.15.89:0/2244125154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:46.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf77fe700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf6ffd700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf8156ab0 0x7fccf814f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf7fff700 1 -- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 msgr2=0x7fccf80b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf7fff700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf7fff700 1 -- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf8156ab0 msgr2=0x7fccf814f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf7fff700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf8156ab0 0x7fccf814f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf7fff700 1 -- 172.21.15.89:0/2244125154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec009550 con 0x7fccf81600b0 2023-07-06T02:23:46.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccf7fff700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80c58b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009fa0 tx=0x7fccf000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccdffff700 1 -- 172.21.15.89:0/2244125154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000ddb0 con 0x7fccf81600b0 2023-07-06T02:23:46.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccdffff700 1 -- 172.21.15.89:0/2244125154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf000f040 con 0x7fccf81600b0 2023-07-06T02:23:46.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf806c6c0 con 0x7fccf81600b0 2023-07-06T02:23:46.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccdffff700 1 -- 172.21.15.89:0/2244125154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0003c60 con 0x7fccf81600b0 2023-07-06T02:23:46.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.067+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf814fb00 con 0x7fccf81600b0 2023-07-06T02:23:46.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.068+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf80b3d30 con 0x7fccf81600b0 2023-07-06T02:23:46.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.069+0000 7fccdffff700 1 -- 172.21.15.89:0/2244125154 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fccf0004030 con 0x7fccf81600b0 2023-07-06T02:23:46.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.069+0000 7fccdffff700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcce406ace0 0x7fcce406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.069+0000 7fccdffff700 1 -- 172.21.15.89:0/2244125154 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fccf0089820 con 0x7fccf81600b0 2023-07-06T02:23:46.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.069+0000 7fccf77fe700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcce406ace0 0x7fcce406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.069+0000 7fccf77fe700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcce406ace0 0x7fcce406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcce80018a0 tx=0x7fcce8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.073+0000 7fccdffff700 1 -- 172.21.15.89:0/2244125154 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fccf80b3d30 con 0x7fccf81600b0 2023-07-06T02:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.259+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fccf80b3d30 con 0x7fccf81600b0 2023-07-06T02:23:46.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.259+0000 7fccdffff700 1 -- 172.21.15.89:0/2244125154 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fccf80b3d30 con 0x7fccf81600b0 2023-07-06T02:23:46.261 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcce406ace0 msgr2=0x7fcce406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcce406ace0 0x7fcce406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcce80018a0 tx=0x7fcce8009040 comp rx=0 tx=0).stop 2023-07-06T02:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 msgr2=0x7fccf80c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80c58b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009fa0 tx=0x7fccf000d3b0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 shutdown_connections 2023-07-06T02:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcce406ace0 0x7fcce406d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf81600b0 0x7fccf80c58b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccf80b19d0 0x7fccf80b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 --2- 172.21.15.89:0/2244125154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf8156ab0 0x7fccf814f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 >> 172.21.15.89:0/2244125154 conn(0x7fccf80cc490 msgr2=0x7fccf80cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:46.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 shutdown_connections 2023-07-06T02:23:46.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.261+0000 7fccfdf06700 1 -- 172.21.15.89:0/2244125154 wait complete. 2023-07-06T02:23:46.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:46.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.396+0000 7f5b6c830700 1 Processor -- start 2023-07-06T02:23:46.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.396+0000 7f5b6c830700 1 -- start start 2023-07-06T02:23:46.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b6c830700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b6815c610 0x7f5b680cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b6c830700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680cb150 con 0x7f5b680b5e50 2023-07-06T02:23:46.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b6c830700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680cb430 con 0x7f5b68070a40 2023-07-06T02:23:46.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b6c830700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680b6310 con 0x7f5b6815c610 2023-07-06T02:23:46.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b6659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b6815c610 0x7f5b680cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b6659c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b6815c610 0x7f5b680cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33870/0 (socket says 172.21.15.89:33870) 2023-07-06T02:23:46.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b6659c700 1 -- 172.21.15.89:0/232120026 learned_addr learned my addr 172.21.15.89:0/232120026 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:46.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.397+0000 7f5b65d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5b680b5e50 0x7f5b68070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58772/0 (socket says 172.21.15.89:58772) 2023-07-06T02:23:46.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6659c700 1 -- 172.21.15.89:0/232120026 >> v1:172.21.15.172:6789/0 conn(0x7f5b68070a40 legacy=0x7f5b680cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:46.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6659c700 1 -- 172.21.15.89:0/232120026 >> v1:172.21.15.89:6789/0 conn(0x7f5b680b5e50 legacy=0x7f5b68070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6659c700 1 -- 172.21.15.89:0/232120026 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680b65f0 con 0x7f5b6815c610 2023-07-06T02:23:46.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6559a700 1 -- 172.21.15.89:0/232120026 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513545708 0 0) 0x7f5b680cb150 con 0x7f5b680b5e50 2023-07-06T02:23:46.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6659c700 1 --2- 172.21.15.89:0/232120026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b6815c610 0x7f5b680cfed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0098f0 tx=0x7f5b5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133aec7fb60e3261 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6559a700 1 -- 172.21.15.89:0/232120026 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c004230 con 0x7f5b6815c610 2023-07-06T02:23:46.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6559a700 1 -- 172.21.15.89:0/232120026 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c004390 con 0x7f5b6815c610 2023-07-06T02:23:46.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6559a700 1 -- 172.21.15.89:0/232120026 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c0036a0 con 0x7f5b6815c610 2023-07-06T02:23:46.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6c830700 1 -- 172.21.15.89:0/232120026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b6815c610 msgr2=0x7f5b680cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.398+0000 7f5b6c830700 1 --2- 172.21.15.89:0/232120026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b6815c610 0x7f5b680cfed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0098f0 tx=0x7f5b5c009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.399+0000 7f5b6c830700 1 -- 172.21.15.89:0/232120026 shutdown_connections 2023-07-06T02:23:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.399+0000 7f5b6c830700 1 --2- 172.21.15.89:0/232120026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b6815c610 0x7f5b680cfed0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.399+0000 7f5b6c830700 1 -- 172.21.15.89:0/232120026 >> 172.21.15.89:0/232120026 conn(0x7f5b680d0b00 msgr2=0x7f5b680d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.399+0000 7f5b6c830700 1 -- 172.21.15.89:0/232120026 shutdown_connections 2023-07-06T02:23:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.399+0000 7f5b6c830700 1 -- 172.21.15.89:0/232120026 wait complete. 2023-07-06T02:23:46.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.400+0000 7f5b6c830700 1 Processor -- start 2023-07-06T02:23:46.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.400+0000 7f5b6c830700 1 -- start start 2023-07-06T02:23:46.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b68070a40 0x7f5b681533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b680b5e50 0x7f5b680c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6815c610 0x7f5b680b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68153900 con 0x7f5b68070a40 2023-07-06T02:23:46.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6815e0f0 con 0x7f5b6815c610 2023-07-06T02:23:46.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6815e3d0 con 0x7f5b680b5e50 2023-07-06T02:23:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b68070a40 0x7f5b681533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b68070a40 0x7f5b681533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33876/0 (socket says 172.21.15.89:33876) 2023-07-06T02:23:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 -- 172.21.15.89:0/2696429613 learned_addr learned my addr 172.21.15.89:0/2696429613 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b65d9b700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b680b5e50 0x7f5b680c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b66d9d700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6815c610 0x7f5b680b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 -- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b680b5e50 msgr2=0x7f5b680c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b680b5e50 0x7f5b680c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 -- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6815c610 msgr2=0x7f5b680b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6815c610 0x7f5b680b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 -- 172.21.15.89:0/2696429613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c009550 con 0x7f5b68070a40 2023-07-06T02:23:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6659c700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b68070a40 0x7f5b681533c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c000c00 tx=0x7f5b5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b56ffd700 1 -- 172.21.15.89:0/2696429613 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00b660 con 0x7f5b68070a40 2023-07-06T02:23:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b56ffd700 1 -- 172.21.15.89:0/2696429613 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c00b7e0 con 0x7f5b68070a40 2023-07-06T02:23:46.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6815e6b0 con 0x7f5b68070a40 2023-07-06T02:23:46.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b56ffd700 1 -- 172.21.15.89:0/2696429613 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00bb90 con 0x7f5b68070a40 2023-07-06T02:23:46.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.401+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b680c46f0 con 0x7f5b68070a40 2023-07-06T02:23:46.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.402+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6815ebd0 con 0x7f5b68070a40 2023-07-06T02:23:46.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.403+0000 7f5b56ffd700 1 -- 172.21.15.89:0/2696429613 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5b5c00bcf0 con 0x7f5b68070a40 2023-07-06T02:23:46.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.404+0000 7f5b56ffd700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b4806ace0 0x7f5b4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.404+0000 7f5b65d9b700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b4806ace0 0x7f5b4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.404+0000 7f5b56ffd700 1 -- 172.21.15.89:0/2696429613 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5b5c08b350 con 0x7f5b68070a40 2023-07-06T02:23:46.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.404+0000 7f5b65d9b700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b4806ace0 0x7f5b4806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b680afda0 tx=0x7f5b58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.408+0000 7f5b56ffd700 1 -- 172.21.15.89:0/2696429613 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5b5c0561c0 con 0x7f5b68070a40 2023-07-06T02:23:46.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.597+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b68074270 con 0x7f5b68070a40 2023-07-06T02:23:46.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.597+0000 7f5b56ffd700 1 -- 172.21.15.89:0/2696429613 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b5c0597e0 con 0x7f5b68070a40 2023-07-06T02:23:46.599 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b4806ace0 msgr2=0x7f5b4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b4806ace0 0x7f5b4806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b680afda0 tx=0x7f5b58009040 comp rx=0 tx=0).stop 2023-07-06T02:23:46.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b68070a40 msgr2=0x7f5b681533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b68070a40 0x7f5b681533c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c000c00 tx=0x7f5b5c004750 comp rx=0 tx=0).stop 2023-07-06T02:23:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 shutdown_connections 2023-07-06T02:23:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6815c610 0x7f5b680b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5b4806ace0 0x7f5b4806d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b68070a40 0x7f5b681533c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 --2- 172.21.15.89:0/2696429613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b680b5e50 0x7f5b680c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 >> 172.21.15.89:0/2696429613 conn(0x7f5b680d0b00 msgr2=0x7f5b68159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:46.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 shutdown_connections 2023-07-06T02:23:46.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.599+0000 7f5b6c830700 1 -- 172.21.15.89:0/2696429613 wait complete. 2023-07-06T02:23:46.613 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.734+0000 7f640abd7700 1 Processor -- start 2023-07-06T02:23:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.734+0000 7f640abd7700 1 -- start start 2023-07-06T02:23:46.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f640abd7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 0x7f64040ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f640abd7700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640406b160 con 0x7f64040c9ee0 2023-07-06T02:23:46.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f640abd7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64040cac10 con 0x7f64040cecf0 2023-07-06T02:23:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f640abd7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64040caa30 con 0x7f640406e960 2023-07-06T02:23:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f6409174700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f64040c9ee0 0x7f64040ca320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58790/0 (socket says 172.21.15.89:58790) 2023-07-06T02:23:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f6409174700 1 -- 172.21.15.89:0/2767447503 learned_addr learned my addr 172.21.15.89:0/2767447503 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952294125 0 0) 0x7f640406b160 con 0x7f64040c9ee0 2023-07-06T02:23:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f40035b0 con 0x7f64040c9ee0 2023-07-06T02:23:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.735+0000 7f6403fff700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 0x7f64040ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365356463 0 0) 0x7f63f40035b0 con 0x7f64040c9ee0 2023-07-06T02:23:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 msgr2=0x7f64040ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 0x7f64040ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 >> v1:172.21.15.172:6789/0 conn(0x7f64040cecf0 legacy=0x7f640406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64040caef0 con 0x7f64040c9ee0 2023-07-06T02:23:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f63fc0019f0 con 0x7f64040c9ee0 2023-07-06T02:23:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 >> v1:172.21.15.89:6789/0 conn(0x7f64040c9ee0 legacy=0x7f64040ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4003a30 0x7f63f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 0x7f63f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4004d90 0x7f63f40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64040caa30 con 0x7f63f4004d90 2023-07-06T02:23:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400bba0 con 0x7f63f4003a30 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400bd30 con 0x7f640406e960 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6409174700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4004d90 0x7f63f40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6408973700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4003a30 0x7f63f4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6403fff700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 0x7f63f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6409174700 1 -- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 msgr2=0x7f63f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6409174700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 0x7f63f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6409174700 1 -- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4003a30 msgr2=0x7f63f4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6409174700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4003a30 0x7f63f4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6409174700 1 -- 172.21.15.89:0/2767447503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40035b0 con 0x7f63f4004d90 2023-07-06T02:23:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6403fff700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f640406e960 0x7f63f4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f6409174700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4004d90 0x7f63f40051e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00a760 tx=0x7f63f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c49d4d935449b1b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc00a250 con 0x7f63f4004d90 2023-07-06T02:23:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0072d0 con 0x7f63f4004d90 2023-07-06T02:23:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.736+0000 7f64037fe700 1 -- 172.21.15.89:0/2767447503 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc007640 con 0x7f63f4004d90 2023-07-06T02:23:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.737+0000 7f640abd7700 1 -- 172.21.15.89:0/2767447503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4004d90 msgr2=0x7f63f40051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.737+0000 7f640abd7700 1 --2- 172.21.15.89:0/2767447503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63f4004d90 0x7f63f40051e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00a760 tx=0x7f63f4005880 comp rx=0 tx=0).stop 2023-07-06T02:23:46.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.737+0000 7f6409174700 1 -- 172.21.15.89:0/2767447503 reap_dead start 2023-07-06T02:23:46.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.737+0000 7f640abd7700 1 -- 172.21.15.89:0/2767447503 shutdown_connections 2023-07-06T02:23:46.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.737+0000 7f640abd7700 1 -- 172.21.15.89:0/2767447503 >> 172.21.15.89:0/2767447503 conn(0x7f64040cf1c0 msgr2=0x7f64040cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:46.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.737+0000 7f640abd7700 1 -- 172.21.15.89:0/2767447503 shutdown_connections 2023-07-06T02:23:46.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.737+0000 7f640abd7700 1 -- 172.21.15.89:0/2767447503 wait complete. 2023-07-06T02:23:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.738+0000 7f640abd7700 1 Processor -- start 2023-07-06T02:23:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.738+0000 7f640abd7700 1 -- start start 2023-07-06T02:23:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f640abd7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4004d90 0x7f64041e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f640abd7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64040c5cd0 0x7f640414edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f640abd7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64040712b0 0x7f6404071750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f640abd7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640414f300 con 0x7f64040712b0 2023-07-06T02:23:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f640abd7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6404071c90 con 0x7f63f4004d90 2023-07-06T02:23:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f640abd7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64040b2190 con 0x7f64040c5cd0 2023-07-06T02:23:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6409174700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64040712b0 0x7f6404071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6409174700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64040712b0 0x7f6404071750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33900/0 (socket says 172.21.15.89:33900) 2023-07-06T02:23:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6409174700 1 -- 172.21.15.89:0/1874576978 learned_addr learned my addr 172.21.15.89:0/1874576978 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6408973700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4004d90 0x7f64041e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6409174700 1 -- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64040c5cd0 msgr2=0x7f640414edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6403fff700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64040c5cd0 0x7f640414edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6409174700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64040c5cd0 0x7f640414edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6409174700 1 -- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4004d90 msgr2=0x7f64041e3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.739+0000 7f6409174700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4004d90 0x7f64041e3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f6409174700 1 -- 172.21.15.89:0/1874576978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640414f710 con 0x7f64040712b0 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f6408973700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4004d90 0x7f64041e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f6409174700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64040712b0 0x7f6404071750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f63fc011000 tx=0x7f63fc001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f64017fa700 1 -- 172.21.15.89:0/1874576978 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc007850 con 0x7f64040712b0 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f64017fa700 1 -- 172.21.15.89:0/1874576978 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0079b0 con 0x7f64040712b0 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f64017fa700 1 -- 172.21.15.89:0/1874576978 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc012640 con 0x7f64040712b0 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64041e3e00 con 0x7f64040712b0 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.740+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64041e43c0 con 0x7f64040712b0 2023-07-06T02:23:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.741+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64040b1ff0 con 0x7f64040712b0 2023-07-06T02:23:46.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.741+0000 7f64017fa700 1 -- 172.21.15.89:0/1874576978 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63fc01a070 con 0x7f64040712b0 2023-07-06T02:23:46.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.741+0000 7f64017fa700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f63e806ac70 0x7f63e806d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:46.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.742+0000 7f6408973700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f63e806ac70 0x7f63e806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:46.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.742+0000 7f64017fa700 1 -- 172.21.15.89:0/1874576978 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f63fc08aef0 con 0x7f64040712b0 2023-07-06T02:23:46.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.742+0000 7f6408973700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f63e806ac70 0x7f63e806d120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000c00 tx=0x7f63f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:46.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.746+0000 7f64017fa700 1 -- 172.21.15.89:0/1874576978 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f64040b1ff0 con 0x7f64040712b0 2023-07-06T02:23:46.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.932+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64040b1ff0 con 0x7f64040712b0 2023-07-06T02:23:46.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.933+0000 7f64017fa700 1 -- 172.21.15.89:0/1874576978 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64040b1ff0 con 0x7f64040712b0 2023-07-06T02:23:46.935 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:46.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f63e806ac70 msgr2=0x7f63e806d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f63e806ac70 0x7f63e806d120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000c00 tx=0x7f63f8009040 comp rx=0 tx=0).stop 2023-07-06T02:23:46.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64040712b0 msgr2=0x7f6404071750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:46.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64040712b0 0x7f6404071750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f63fc011000 tx=0x7f63fc001cb0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 shutdown_connections 2023-07-06T02:23:46.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f63e806ac70 0x7f63e806d120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64040712b0 0x7f6404071750 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f4004d90 0x7f64041e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 --2- 172.21.15.89:0/1874576978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64040c5cd0 0x7f640414edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:46.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 >> 172.21.15.89:0/1874576978 conn(0x7f64040cf1c0 msgr2=0x7f64041557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:46.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 shutdown_connections 2023-07-06T02:23:46.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:46.935+0000 7f640abd7700 1 -- 172.21.15.89:0/1874576978 wait complete. 2023-07-06T02:23:46.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:47.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.070+0000 7fcb6d6d7700 1 Processor -- start 2023-07-06T02:23:47.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.070+0000 7fcb6d6d7700 1 -- start start 2023-07-06T02:23:47.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb6d6d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 0x7fcb6806aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb6d6d7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb6806b610 0x7fcb6806c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb6d6d7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb680ceda0 0x7fcb680d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb6d6d7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680cd240 con 0x7fcb6806b610 2023-07-06T02:23:47.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb6d6d7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680cf280 con 0x7fcb680ceda0 2023-07-06T02:23:47.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb6d6d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680cd490 con 0x7fcb6806bdb0 2023-07-06T02:23:47.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb6806b610 0x7fcb6806c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb6806b610 0x7fcb6806c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33906/0 (socket says 172.21.15.89:33906) 2023-07-06T02:23:47.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 -- 172.21.15.89:0/501434185 learned_addr learned my addr 172.21.15.89:0/501434185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:47.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb677fe700 1 --2- 172.21.15.89:0/501434185 >> v2:172.21.15.172:3300/0 conn(0x7fcb680ceda0 0x7fcb680d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 -- 172.21.15.89:0/501434185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 msgr2=0x7fcb6806aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:47.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 --2- 172.21.15.89:0/501434185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 0x7fcb6806aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 -- 172.21.15.89:0/501434185 >> v2:172.21.15.172:3300/0 conn(0x7fcb680ceda0 msgr2=0x7fcb680d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:47.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 --2- 172.21.15.89:0/501434185 >> v2:172.21.15.172:3300/0 conn(0x7fcb680ceda0 0x7fcb680d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 -- 172.21.15.89:0/501434185 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680cd770 con 0x7fcb6806b610 2023-07-06T02:23:47.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.071+0000 7fcb667fc700 1 --2- 172.21.15.89:0/501434185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6806b610 0x7fcb6806c180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb6806d050 tx=0x7fcb58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58ba5a31023a53d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb65ffb700 1 -- 172.21.15.89:0/501434185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800b870 con 0x7fcb6806b610 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb65ffb700 1 -- 172.21.15.89:0/501434185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5800b9d0 con 0x7fcb6806b610 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb65ffb700 1 -- 172.21.15.89:0/501434185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800bd60 con 0x7fcb6806b610 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/501434185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6806b610 msgr2=0x7fcb6806c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/501434185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6806b610 0x7fcb6806c180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb6806d050 tx=0x7fcb58009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/501434185 shutdown_connections 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/501434185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 0x7fcb6806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/501434185 >> v2:172.21.15.172:3300/0 conn(0x7fcb680ceda0 0x7fcb680d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/501434185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6806b610 0x7fcb6806c180 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/501434185 >> 172.21.15.89:0/501434185 conn(0x7fcb680b5390 msgr2=0x7fcb680d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:47.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/501434185 shutdown_connections 2023-07-06T02:23:47.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.072+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/501434185 wait complete. 2023-07-06T02:23:47.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 Processor -- start 2023-07-06T02:23:47.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 -- start start 2023-07-06T02:23:47.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 0x7fcb68145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb680ceda0 0x7fcb680cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb680cc880 0x7fcb680b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680cb2a0 con 0x7fcb680cc880 2023-07-06T02:23:47.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680cb580 con 0x7fcb680ceda0 2023-07-06T02:23:47.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb6d6d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680cb860 con 0x7fcb6806bdb0 2023-07-06T02:23:47.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb680cc880 0x7fcb680b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb680cc880 0x7fcb680b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33914/0 (socket says 172.21.15.89:33914) 2023-07-06T02:23:47.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.074+0000 7fcb677fe700 1 -- 172.21.15.89:0/3017897982 learned_addr learned my addr 172.21.15.89:0/3017897982 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:47.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb667fc700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb680ceda0 0x7fcb680cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb66ffd700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 0x7fcb68145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb677fe700 1 -- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 msgr2=0x7fcb68145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb677fe700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 0x7fcb68145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb677fe700 1 -- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb680ceda0 msgr2=0x7fcb680cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb677fe700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb680ceda0 0x7fcb680cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb677fe700 1 -- 172.21.15.89:0/3017897982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58009550 con 0x7fcb680cc880 2023-07-06T02:23:47.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb667fc700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb680ceda0 0x7fcb680cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:47.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb677fe700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb680cc880 0x7fcb680b6a70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000cc30 tx=0x7fcb6000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb4f7fe700 1 -- 172.21.15.89:0/3017897982 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000ab70 con 0x7fcb680cc880 2023-07-06T02:23:47.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb4f7fe700 1 -- 172.21.15.89:0/3017897982 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6000acd0 con 0x7fcb680cc880 2023-07-06T02:23:47.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb680cbb40 con 0x7fcb680cc880 2023-07-06T02:23:47.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb4f7fe700 1 -- 172.21.15.89:0/3017897982 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60003c40 con 0x7fcb680cc880 2023-07-06T02:23:47.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.075+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb68146360 con 0x7fcb680cc880 2023-07-06T02:23:47.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.076+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb680cc060 con 0x7fcb680cc880 2023-07-06T02:23:47.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.076+0000 7fcb4f7fe700 1 -- 172.21.15.89:0/3017897982 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb6000f3f0 con 0x7fcb680cc880 2023-07-06T02:23:47.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.077+0000 7fcb4f7fe700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcb5406ace0 0x7fcb5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.077+0000 7fcb4f7fe700 1 -- 172.21.15.89:0/3017897982 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcb60089a30 con 0x7fcb680cc880 2023-07-06T02:23:47.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.077+0000 7fcb66ffd700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcb5406ace0 0x7fcb5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.077+0000 7fcb66ffd700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcb5406ace0 0x7fcb5406d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c006fd0 tx=0x7fcb5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.081+0000 7fcb4f7fe700 1 -- 172.21.15.89:0/3017897982 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb680cc060 con 0x7fcb680cc880 2023-07-06T02:23:47.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.267+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb680cc060 con 0x7fcb680cc880 2023-07-06T02:23:47.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.268+0000 7fcb4f7fe700 1 -- 172.21.15.89:0/3017897982 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb680cc060 con 0x7fcb680cc880 2023-07-06T02:23:47.270 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:47.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcb5406ace0 msgr2=0x7fcb5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcb5406ace0 0x7fcb5406d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c006fd0 tx=0x7fcb5c008040 comp rx=0 tx=0).stop 2023-07-06T02:23:47.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb680cc880 msgr2=0x7fcb680b6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb680cc880 0x7fcb680b6a70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000cc30 tx=0x7fcb6000a1f0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 shutdown_connections 2023-07-06T02:23:47.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcb5406ace0 0x7fcb5406d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb680cc880 0x7fcb680b6a70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb6806bdb0 0x7fcb68145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 --2- 172.21.15.89:0/3017897982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb680ceda0 0x7fcb680cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 >> 172.21.15.89:0/3017897982 conn(0x7fcb680b5390 msgr2=0x7fcb680cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:47.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 shutdown_connections 2023-07-06T02:23:47.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.270+0000 7fcb6d6d7700 1 -- 172.21.15.89:0/3017897982 wait complete. 2023-07-06T02:23:47.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:47.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.405+0000 7fdcc5ad5700 1 Processor -- start 2023-07-06T02:23:47.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.405+0000 7fdcc5ad5700 1 -- start start 2023-07-06T02:23:47.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.405+0000 7fdcc5ad5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdcc00b1490 0x7fdcc00b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.405+0000 7fdcc5ad5700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc00cab50 con 0x7fdcc006bfa0 2023-07-06T02:23:47.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.405+0000 7fdcc5ad5700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc015db70 con 0x7fdcc015c690 2023-07-06T02:23:47.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.405+0000 7fdcc5ad5700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc015de50 con 0x7fdcc00b1490 2023-07-06T02:23:47.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbeffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdcc00b1490 0x7fdcc00b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdcc006bfa0 0x7fdcc006c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58822/0 (socket says 172.21.15.89:58822) 2023-07-06T02:23:47.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbeffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdcc00b1490 0x7fdcc00b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33918/0 (socket says 172.21.15.89:33918) 2023-07-06T02:23:47.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbffff700 1 -- 172.21.15.89:0/1447637595 learned_addr learned my addr 172.21.15.89:0/1447637595 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:47.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbeffd700 1 -- 172.21.15.89:0/1447637595 >> v1:172.21.15.172:6789/0 conn(0x7fdcc015c690 legacy=0x7fdcc00cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbeffd700 1 -- 172.21.15.89:0/1447637595 >> v1:172.21.15.89:6789/0 conn(0x7fdcc006bfa0 legacy=0x7fdcc006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbeffd700 1 -- 172.21.15.89:0/1447637595 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc00cad80 con 0x7fdcc00b1490 2023-07-06T02:23:47.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbe7fc700 1 -- 172.21.15.89:0/1447637595 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599237152 0 0) 0x7fdcc00cab50 con 0x7fdcc006bfa0 2023-07-06T02:23:47.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbeffd700 1 --2- 172.21.15.89:0/1447637595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc00b1490 0x7fdcc00b1910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009840 tx=0x7fdcb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b558e872ebfacf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbe7fc700 1 -- 172.21.15.89:0/1447637595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0011830 con 0x7fdcc00b1490 2023-07-06T02:23:47.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbe7fc700 1 -- 172.21.15.89:0/1447637595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb0011990 con 0x7fdcc00b1490 2023-07-06T02:23:47.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.406+0000 7fdcbe7fc700 1 -- 172.21.15.89:0/1447637595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0011d20 con 0x7fdcc00b1490 2023-07-06T02:23:47.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.407+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/1447637595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc00b1490 msgr2=0x7fdcc00b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.407+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/1447637595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc00b1490 0x7fdcc00b1910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009840 tx=0x7fdcb0009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:47.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.407+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/1447637595 shutdown_connections 2023-07-06T02:23:47.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.407+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/1447637595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc00b1490 0x7fdcc00b1910 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.407+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/1447637595 >> 172.21.15.89:0/1447637595 conn(0x7fdcc00cc490 msgr2=0x7fdcc00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:47.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.407+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/1447637595 shutdown_connections 2023-07-06T02:23:47.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.407+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/1447637595 wait complete. 2023-07-06T02:23:47.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.408+0000 7fdcc5ad5700 1 Processor -- start 2023-07-06T02:23:47.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.408+0000 7fdcc5ad5700 1 -- start start 2023-07-06T02:23:47.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc006bfa0 0x7fdcc00c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcc00b1490 0x7fdcc014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc015c690 0x7fdcc00b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc01593a0 con 0x7fdcc006bfa0 2023-07-06T02:23:47.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00c5b60 con 0x7fdcc015c690 2023-07-06T02:23:47.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00c4f50 con 0x7fdcc00b1490 2023-07-06T02:23:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc006bfa0 0x7fdcc00c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc006bfa0 0x7fdcc00c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33922/0 (socket says 172.21.15.89:33922) 2023-07-06T02:23:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 -- 172.21.15.89:0/2728674576 learned_addr learned my addr 172.21.15.89:0/2728674576 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbffff700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc015c690 0x7fdcc00b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbeffd700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcc00b1490 0x7fdcc014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 -- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcc00b1490 msgr2=0x7fdcc014dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcc00b1490 0x7fdcc014dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 -- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc015c690 msgr2=0x7fdcc00b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc015c690 0x7fdcc00b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 -- 172.21.15.89:0/2728674576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb00094a0 con 0x7fdcc006bfa0 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcbf7fe700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc006bfa0 0x7fdcc00c5ff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c4e0 tx=0x7fdcb400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdca7fff700 1 -- 172.21.15.89:0/2728674576 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40052d0 con 0x7fdcc006bfa0 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdca7fff700 1 -- 172.21.15.89:0/2728674576 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4005430 con 0x7fdcc006bfa0 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc00c5230 con 0x7fdcc006bfa0 2023-07-06T02:23:47.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdca7fff700 1 -- 172.21.15.89:0/2728674576 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40038c0 con 0x7fdcc006bfa0 2023-07-06T02:23:47.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.409+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc00c57f0 con 0x7fdcc006bfa0 2023-07-06T02:23:47.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.410+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc00c5e40 con 0x7fdcc006bfa0 2023-07-06T02:23:47.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.411+0000 7fdca7fff700 1 -- 172.21.15.89:0/2728674576 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdcb4003a20 con 0x7fdcc006bfa0 2023-07-06T02:23:47.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.411+0000 7fdca7fff700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdcac06ace0 0x7fdcac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.411+0000 7fdca7fff700 1 -- 172.21.15.89:0/2728674576 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdcb4088cd0 con 0x7fdcc006bfa0 2023-07-06T02:23:47.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.411+0000 7fdcbeffd700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdcac06ace0 0x7fdcac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.412+0000 7fdcbeffd700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdcac06ace0 0x7fdcac06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00c99d0 tx=0x7fdcb001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.416+0000 7fdca7fff700 1 -- 172.21.15.89:0/2728674576 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdcb4053a50 con 0x7fdcc006bfa0 2023-07-06T02:23:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.602+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdcc00cbf40 con 0x7fdcc006bfa0 2023-07-06T02:23:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.602+0000 7fdca7fff700 1 -- 172.21.15.89:0/2728674576 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdcb4017020 con 0x7fdcc006bfa0 2023-07-06T02:23:47.605 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:47.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdcac06ace0 msgr2=0x7fdcac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdcac06ace0 0x7fdcac06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00c99d0 tx=0x7fdcb001c040 comp rx=0 tx=0).stop 2023-07-06T02:23:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc006bfa0 msgr2=0x7fdcc00c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc006bfa0 0x7fdcc00c5ff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c4e0 tx=0x7fdcb400c8a0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 shutdown_connections 2023-07-06T02:23:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc015c690 0x7fdcc00b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdcac06ace0 0x7fdcac06d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcc006bfa0 0x7fdcc00c5ff0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 --2- 172.21.15.89:0/2728674576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcc00b1490 0x7fdcc014dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 >> 172.21.15.89:0/2728674576 conn(0x7fdcc00cc490 msgr2=0x7fdcc015ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:47.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.604+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 shutdown_connections 2023-07-06T02:23:47.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.605+0000 7fdcc5ad5700 1 -- 172.21.15.89:0/2728674576 wait complete. 2023-07-06T02:23:47.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T02:23:47.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.736+0000 7fe120709700 1 Processor -- start 2023-07-06T02:23:47.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.737+0000 7fe120709700 1 -- start start 2023-07-06T02:23:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.737+0000 7fe120709700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe1181600b0 0x7fe1180b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.737+0000 7fe120709700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1180b19d0 0x7fe1180cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.737+0000 7fe120709700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1180cb3d0 con 0x7fe118156ab0 2023-07-06T02:23:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.737+0000 7fe120709700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1180b1dd0 con 0x7fe1180b19d0 2023-07-06T02:23:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.737+0000 7fe120709700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe118156e80 con 0x7fe1181600b0 2023-07-06T02:23:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1180b19d0 0x7fe1180cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1180b19d0 0x7fe1180cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33934/0 (socket says 172.21.15.89:33934) 2023-07-06T02:23:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 -- 172.21.15.89:0/4144778614 learned_addr learned my addr 172.21.15.89:0/4144778614 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11e4a5700 1 --1- 172.21.15.89:0/4144778614 >> v1:172.21.15.89:6789/0 conn(0x7fe118156ab0 0x7fe118160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58832/0 (socket says 172.21.15.89:58832) 2023-07-06T02:23:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11dca4700 1 --2- 172.21.15.89:0/4144778614 >> v2:172.21.15.172:3300/0 conn(0x7fe1181600b0 0x7fe1180b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 -- 172.21.15.89:0/4144778614 >> v2:172.21.15.172:3300/0 conn(0x7fe1181600b0 msgr2=0x7fe1180b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 --2- 172.21.15.89:0/4144778614 >> v2:172.21.15.172:3300/0 conn(0x7fe1181600b0 0x7fe1180b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 -- 172.21.15.89:0/4144778614 >> v1:172.21.15.89:6789/0 conn(0x7fe118156ab0 legacy=0x7fe118160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 -- 172.21.15.89:0/4144778614 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1180ca370 con 0x7fe1180b19d0 2023-07-06T02:23:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11d4a3700 1 -- 172.21.15.89:0/4144778614 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278956280 0 0) 0x7fe1180cb3d0 con 0x7fe118156ab0 2023-07-06T02:23:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11eca6700 1 --2- 172.21.15.89:0/4144778614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1180b19d0 0x7fe1180cd700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe11815fef0 tx=0x7fe11000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111fa555a4c0de6e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11d4a3700 1 -- 172.21.15.89:0/4144778614 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11000e070 con 0x7fe1180b19d0 2023-07-06T02:23:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11d4a3700 1 -- 172.21.15.89:0/4144778614 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11000b9c0 con 0x7fe1180b19d0 2023-07-06T02:23:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.738+0000 7fe11d4a3700 1 -- 172.21.15.89:0/4144778614 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11000bd70 con 0x7fe1180b19d0 2023-07-06T02:23:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 -- 172.21.15.89:0/4144778614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1180b19d0 msgr2=0x7fe1180cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 --2- 172.21.15.89:0/4144778614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1180b19d0 0x7fe1180cd700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe11815fef0 tx=0x7fe11000b330 comp rx=0 tx=0).stop 2023-07-06T02:23:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 -- 172.21.15.89:0/4144778614 shutdown_connections 2023-07-06T02:23:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 --2- 172.21.15.89:0/4144778614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1180b19d0 0x7fe1180cd700 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 --2- 172.21.15.89:0/4144778614 >> v2:172.21.15.172:3300/0 conn(0x7fe1181600b0 0x7fe1180b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 -- 172.21.15.89:0/4144778614 >> 172.21.15.89:0/4144778614 conn(0x7fe1180cc490 msgr2=0x7fe1180cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 -- 172.21.15.89:0/4144778614 shutdown_connections 2023-07-06T02:23:47.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.739+0000 7fe120709700 1 -- 172.21.15.89:0/4144778614 wait complete. 2023-07-06T02:23:47.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.740+0000 7fe120709700 1 Processor -- start 2023-07-06T02:23:47.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.740+0000 7fe120709700 1 -- start start 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1180b19d0 0x7fe1180abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe118156ab0 0x7fe11815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1181600b0 0x7fe11815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181f5ef0 con 0x7fe118156ab0 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180adad0 con 0x7fe1181600b0 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180addb0 con 0x7fe1180b19d0 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe118156ab0 0x7fe11815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe118156ab0 0x7fe11815cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33938/0 (socket says 172.21.15.89:33938) 2023-07-06T02:23:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 -- 172.21.15.89:0/1603944196 learned_addr learned my addr 172.21.15.89:0/1603944196 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11e4a5700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1180b19d0 0x7fe1180abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11eca6700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1181600b0 0x7fe11815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 -- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1180b19d0 msgr2=0x7fe1180abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1180b19d0 0x7fe1180abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 -- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1181600b0 msgr2=0x7fe11815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1181600b0 0x7fe11815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 -- 172.21.15.89:0/1603944196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110009b30 con 0x7fe118156ab0 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11e4a5700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1180b19d0 0x7fe1180abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe11dca4700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe118156ab0 0x7fe11815cec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d800 tx=0x7fe11400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe10effd700 1 -- 172.21.15.89:0/1603944196 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140125f0 con 0x7fe118156ab0 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe10effd700 1 -- 172.21.15.89:0/1603944196 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114012750 con 0x7fe118156ab0 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1180ae090 con 0x7fe118156ab0 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe10effd700 1 -- 172.21.15.89:0/1603944196 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114012b00 con 0x7fe118156ab0 2023-07-06T02:23:47.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.741+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1180ac3f0 con 0x7fe118156ab0 2023-07-06T02:23:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.742+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe118074270 con 0x7fe118156ab0 2023-07-06T02:23:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.743+0000 7fe10effd700 1 -- 172.21.15.89:0/1603944196 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe114012c60 con 0x7fe118156ab0 2023-07-06T02:23:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.743+0000 7fe10effd700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0fc06ace0 0x7fe0fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.743+0000 7fe10effd700 1 -- 172.21.15.89:0/1603944196 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe114089b40 con 0x7fe118156ab0 2023-07-06T02:23:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.743+0000 7fe11e4a5700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0fc06ace0 0x7fe0fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:47.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.744+0000 7fe11e4a5700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0fc06ace0 0x7fe0fc06d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe108000f60 tx=0x7fe108007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.747+0000 7fe10effd700 1 -- 172.21.15.89:0/1603944196 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe1140548c0 con 0x7fe118156ab0 2023-07-06T02:23:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.947+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe1180b5990 con 0x7fe118156ab0 2023-07-06T02:23:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.948+0000 7fe10effd700 1 -- 172.21.15.89:0/1603944196 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe114057ee0 con 0x7fe118156ab0 2023-07-06T02:23:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0fc06ace0 msgr2=0x7fe0fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0fc06ace0 0x7fe0fc06d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe108000f60 tx=0x7fe108007ca0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe118156ab0 msgr2=0x7fe11815cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe118156ab0 0x7fe11815cec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d800 tx=0x7fe11400dbc0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 shutdown_connections 2023-07-06T02:23:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1181600b0 0x7fe11815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1180b19d0 0x7fe1180abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0fc06ace0 0x7fe0fc06d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 --2- 172.21.15.89:0/1603944196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe118156ab0 0x7fe11815cec0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 >> 172.21.15.89:0/1603944196 conn(0x7fe1180cc490 msgr2=0x7fe11815e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 shutdown_connections 2023-07-06T02:23:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:47.950+0000 7fe120709700 1 -- 172.21.15.89:0/1603944196 wait complete. 2023-07-06T02:23:47.954 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-07-06T02:23:47.963 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-06T02:23:47.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:47.964 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-07-06T02:23:48.013 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:23:48.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:48.014 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T02:23:48.042 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:48.042 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:48.059 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:48.059 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:48.115 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:23:48.116 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:48.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:48.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.331+0000 7fd2fa237700 1 Processor -- start 2023-07-06T02:23:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.331+0000 7fd2fa237700 1 -- start start 2023-07-06T02:23:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2fa237700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd2f41600b0 0x7fd2f40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2fa237700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd2f40b19d0 0x7fd2f40cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2fa237700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f40cb3d0 con 0x7fd2f4156ab0 2023-07-06T02:23:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2fa237700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f40b1dd0 con 0x7fd2f41600b0 2023-07-06T02:23:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2fa237700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f4156e80 con 0x7fd2f40b19d0 2023-07-06T02:23:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd2f41600b0 0x7fd2f40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd2f4156ab0 0x7fd2f4160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58846/0 (socket says 172.21.15.89:58846) 2023-07-06T02:23:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f37fe700 1 -- 172.21.15.89:0/2178852825 learned_addr learned my addr 172.21.15.89:0/2178852825 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f2ffd700 1 -- 172.21.15.89:0/2178852825 >> v2:172.21.15.172:3300/0 conn(0x7fd2f40b19d0 msgr2=0x7fd2f40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f3fff700 1 --2- 172.21.15.89:0/2178852825 >> v2:172.21.15.172:3300/0 conn(0x7fd2f40b19d0 0x7fd2f40cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f2ffd700 1 --2- 172.21.15.89:0/2178852825 >> v2:172.21.15.172:3300/0 conn(0x7fd2f40b19d0 0x7fd2f40cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f2ffd700 1 -- 172.21.15.89:0/2178852825 >> v1:172.21.15.89:6789/0 conn(0x7fd2f4156ab0 legacy=0x7fd2f4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f27fc700 1 -- 172.21.15.89:0/2178852825 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805679409 0 0) 0x7fd2f40cb3d0 con 0x7fd2f4156ab0 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f2ffd700 1 -- 172.21.15.89:0/2178852825 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40ca370 con 0x7fd2f41600b0 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f3fff700 1 --2- 172.21.15.89:0/2178852825 >> v2:172.21.15.172:3300/0 conn(0x7fd2f40b19d0 0x7fd2f40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f2ffd700 1 --2- 172.21.15.89:0/2178852825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b1490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40c9ff0 tx=0x7fd2e8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7eddfdd24547c82 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f27fc700 1 -- 172.21.15.89:0/2178852825 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8011830 con 0x7fd2f41600b0 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f27fc700 1 -- 172.21.15.89:0/2178852825 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8011990 con 0x7fd2f41600b0 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.332+0000 7fd2f27fc700 1 -- 172.21.15.89:0/2178852825 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8011d20 con 0x7fd2f41600b0 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 -- 172.21.15.89:0/2178852825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 msgr2=0x7fd2f40b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 --2- 172.21.15.89:0/2178852825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b1490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40c9ff0 tx=0x7fd2e8009b20 comp rx=0 tx=0).stop 2023-07-06T02:23:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 -- 172.21.15.89:0/2178852825 shutdown_connections 2023-07-06T02:23:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 --2- 172.21.15.89:0/2178852825 >> v2:172.21.15.172:3300/0 conn(0x7fd2f40b19d0 0x7fd2f40cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 --2- 172.21.15.89:0/2178852825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b1490 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 -- 172.21.15.89:0/2178852825 >> 172.21.15.89:0/2178852825 conn(0x7fd2f40cc490 msgr2=0x7fd2f40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 -- 172.21.15.89:0/2178852825 shutdown_connections 2023-07-06T02:23:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.333+0000 7fd2fa237700 1 -- 172.21.15.89:0/2178852825 wait complete. 2023-07-06T02:23:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.334+0000 7fd2fa237700 1 Processor -- start 2023-07-06T02:23:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.334+0000 7fd2fa237700 1 -- start start 2023-07-06T02:23:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2fa237700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2f40b19d0 0x7fd2f40c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2fa237700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f4156ab0 0x7fd2f40b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2fa237700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2fa237700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40b42b0 con 0x7fd2f41600b0 2023-07-06T02:23:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2fa237700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40b4590 con 0x7fd2f4156ab0 2023-07-06T02:23:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2fa237700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40b4870 con 0x7fd2f40b19d0 2023-07-06T02:23:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33954/0 (socket says 172.21.15.89:33954) 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 -- 172.21.15.89:0/726864598 learned_addr learned my addr 172.21.15.89:0/726864598 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f2ffd700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f4156ab0 0x7fd2f40b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 -- 172.21.15.89:0/726864598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2f40b19d0 msgr2=0x7fd2f40c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2f40b19d0 0x7fd2f40c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 -- 172.21.15.89:0/726864598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f4156ab0 msgr2=0x7fd2f40b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f4156ab0 0x7fd2f40b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.335+0000 7fd2f3fff700 1 -- 172.21.15.89:0/726864598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80094a0 con 0x7fd2f41600b0 2023-07-06T02:23:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.336+0000 7fd2f3fff700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b3270 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00cef0 tx=0x7fd2ec00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.336+0000 7fd2dbfff700 1 -- 172.21.15.89:0/726864598 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00ae50 con 0x7fd2f41600b0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.336+0000 7fd2dbfff700 1 -- 172.21.15.89:0/726864598 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec015440 con 0x7fd2f41600b0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.336+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f40b4b50 con 0x7fd2f41600b0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.336+0000 7fd2dbfff700 1 -- 172.21.15.89:0/726864598 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0157f0 con 0x7fd2f41600b0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.336+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f40b5110 con 0x7fd2f41600b0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.337+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f40b5330 con 0x7fd2f41600b0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.337+0000 7fd2dbfff700 1 -- 172.21.15.89:0/726864598 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2ec00d430 con 0x7fd2f41600b0 2023-07-06T02:23:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.337+0000 7fd2dbfff700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e006ace0 0x7fd2e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.337+0000 7fd2dbfff700 1 -- 172.21.15.89:0/726864598 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd2ec089e10 con 0x7fd2f41600b0 2023-07-06T02:23:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.338+0000 7fd2f37fe700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e006ace0 0x7fd2e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.338+0000 7fd2f37fe700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e006ace0 0x7fd2e006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40011e0 tx=0x7fd2e40079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.342+0000 7fd2dbfff700 1 -- 172.21.15.89:0/726864598 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd2ec054b90 con 0x7fd2f41600b0 2023-07-06T02:23:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.538+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2f4074270 con 0x7fd2f41600b0 2023-07-06T02:23:48.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.539+0000 7fd2dbfff700 1 -- 172.21.15.89:0/726864598 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fd2ec01a090 con 0x7fd2f41600b0 2023-07-06T02:23:48.541 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:48.541 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:23:41.389931+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:48.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e006ace0 msgr2=0x7fd2e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e006ace0 0x7fd2e006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40011e0 tx=0x7fd2e40079b0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 msgr2=0x7fd2f40b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b3270 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00cef0 tx=0x7fd2ec00a340 comp rx=0 tx=0).stop 2023-07-06T02:23:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 shutdown_connections 2023-07-06T02:23:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd2e006ace0 0x7fd2e006d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2f41600b0 0x7fd2f40b3270 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2f40b19d0 0x7fd2f40c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 --2- 172.21.15.89:0/726864598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2f4156ab0 0x7fd2f40b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.541+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 >> 172.21.15.89:0/726864598 conn(0x7fd2f40cc490 msgr2=0x7fd2f415dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:48.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.542+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 shutdown_connections 2023-07-06T02:23:48.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.542+0000 7fd2fa237700 1 -- 172.21.15.89:0/726864598 wait complete. 2023-07-06T02:23:48.556 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:48.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:48.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.677+0000 7f9b0b0b6700 1 Processor -- start 2023-07-06T02:23:48.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.677+0000 7f9b0b0b6700 1 -- start start 2023-07-06T02:23:48.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.678+0000 7f9b0b0b6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9b040705a0 0x7f9b040ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.678+0000 7f9b0b0b6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 0x7f9b04157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.678+0000 7f9b0b0b6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0406e110 con 0x7f9b040cd180 2023-07-06T02:23:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.678+0000 7f9b0b0b6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0406e3f0 con 0x7f9b040705a0 2023-07-06T02:23:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.678+0000 7f9b0b0b6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0406e6d0 con 0x7f9b0406dc10 2023-07-06T02:23:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.678+0000 7f9b03fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9b040cd180 0x7f9b0415a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58860/0 (socket says 172.21.15.89:58860) 2023-07-06T02:23:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.678+0000 7f9b03fff700 1 -- 172.21.15.89:0/1588380904 learned_addr learned my addr 172.21.15.89:0/1588380904 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928531072 0 0) 0x7f9b0406e110 con 0x7f9b040cd180 2023-07-06T02:23:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b09653700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 0x7f9b04157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b08e52700 1 --2- 172.21.15.89:0/1588380904 >> v2:172.21.15.172:3300/0 conn(0x7f9b040705a0 0x7f9b040ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af40035b0 con 0x7f9b040cd180 2023-07-06T02:23:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 169285919 0 0) 0x7f9af40035b0 con 0x7f9b040cd180 2023-07-06T02:23:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 msgr2=0x7f9b04157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 0x7f9b04157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 >> v2:172.21.15.172:3300/0 conn(0x7f9b040705a0 msgr2=0x7f9b040ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 --2- 172.21.15.89:0/1588380904 >> v2:172.21.15.172:3300/0 conn(0x7f9b040705a0 0x7f9b040ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0406e9b0 con 0x7f9b040cd180 2023-07-06T02:23:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f9af0001670 con 0x7f9b040cd180 2023-07-06T02:23:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 >> v1:172.21.15.89:6789/0 conn(0x7f9b040cd180 legacy=0x7f9b0415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 0x7f9af4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af4003fd0 0x7f9af4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af4004d90 0x7f9af4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0406e3f0 con 0x7f9af4003fd0 2023-07-06T02:23:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0406e6d0 con 0x7f9af4004d90 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af400ba10 con 0x7f9b0406dc10 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b03fff700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af4003fd0 0x7f9af4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b08e52700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 0x7f9af4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.679+0000 7f9b09653700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af4004d90 0x7f9af4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b03fff700 1 -- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 msgr2=0x7f9af4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b03fff700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b0406dc10 0x7f9af4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b03fff700 1 -- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af4004d90 msgr2=0x7f9af4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b03fff700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af4004d90 0x7f9af4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b03fff700 1 -- 172.21.15.89:0/1588380904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af40035b0 con 0x7f9af4003fd0 2023-07-06T02:23:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b03fff700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af4003fd0 0x7f9af4004850 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c210 tx=0x7f9af4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1fdbb8d8a7ed40 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000cd50 con 0x7f9af4003fd0 2023-07-06T02:23:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0005960 con 0x7f9af4003fd0 2023-07-06T02:23:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b037fe700 1 -- 172.21.15.89:0/1588380904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0005d10 con 0x7f9af4003fd0 2023-07-06T02:23:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/1588380904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af4003fd0 msgr2=0x7f9af4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b0b0b6700 1 --2- 172.21.15.89:0/1588380904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9af4003fd0 0x7f9af4004850 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c210 tx=0x7f9af4005880 comp rx=0 tx=0).stop 2023-07-06T02:23:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b09653700 1 -- 172.21.15.89:0/1588380904 reap_dead start 2023-07-06T02:23:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/1588380904 shutdown_connections 2023-07-06T02:23:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/1588380904 >> 172.21.15.89:0/1588380904 conn(0x7f9b040d0b00 msgr2=0x7f9b040d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/1588380904 shutdown_connections 2023-07-06T02:23:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.680+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/1588380904 wait complete. 2023-07-06T02:23:48.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.681+0000 7f9b0b0b6700 1 Processor -- start 2023-07-06T02:23:48.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b0b0b6700 1 -- start start 2023-07-06T02:23:48.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b0b0b6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9af4003fd0 0x7f9b040c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b0b0b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041e4c30 0x7f9b041f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b0b0b6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b041f5f00 0x7f9b040c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b0b0b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041f6370 con 0x7f9b041e4c30 2023-07-06T02:23:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b0b0b6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041e4470 con 0x7f9b041f5f00 2023-07-06T02:23:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b0b0b6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040c4b70 con 0x7f9af4003fd0 2023-07-06T02:23:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.682+0000 7f9b03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041e4c30 0x7f9b041f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041e4c30 0x7f9b041f59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33974/0 (socket says 172.21.15.89:33974) 2023-07-06T02:23:48.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 -- 172.21.15.89:0/2580687393 learned_addr learned my addr 172.21.15.89:0/2580687393 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:48.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b09653700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b041f5f00 0x7f9b040c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b08e52700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9af4003fd0 0x7f9b040c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 -- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9af4003fd0 msgr2=0x7f9b040c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9af4003fd0 0x7f9b040c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 -- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b041f5f00 msgr2=0x7f9b040c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b041f5f00 0x7f9b040c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 -- 172.21.15.89:0/2580687393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b041f6860 con 0x7f9b041e4c30 2023-07-06T02:23:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b03fff700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041e4c30 0x7f9b041f59c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c800 tx=0x7f9af0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b017fa700 1 -- 172.21.15.89:0/2580687393 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0022070 con 0x7f9b041e4c30 2023-07-06T02:23:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b017fa700 1 -- 172.21.15.89:0/2580687393 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0005ce0 con 0x7f9b041e4c30 2023-07-06T02:23:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b017fa700 1 -- 172.21.15.89:0/2580687393 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0004a90 con 0x7f9b041e4c30 2023-07-06T02:23:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b041f6580 con 0x7f9b041e4c30 2023-07-06T02:23:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.683+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b041e5b90 con 0x7f9b041e4c30 2023-07-06T02:23:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.684+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b040cc260 con 0x7f9b041e4c30 2023-07-06T02:23:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.684+0000 7f9b017fa700 1 -- 172.21.15.89:0/2580687393 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9af0004bf0 con 0x7f9b041e4c30 2023-07-06T02:23:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.685+0000 7f9b017fa700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9ae806ad10 0x7f9ae806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.685+0000 7f9b017fa700 1 -- 172.21.15.89:0/2580687393 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9af0089e10 con 0x7f9b041e4c30 2023-07-06T02:23:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.685+0000 7f9b08e52700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9ae806ad10 0x7f9ae806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.685+0000 7f9b08e52700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9ae806ad10 0x7f9ae806d1c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009c60 tx=0x7f9af8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.689+0000 7f9b017fa700 1 -- 172.21.15.89:0/2580687393 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9af0054c20 con 0x7f9b041e4c30 2023-07-06T02:23:48.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.875+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9b04074270 con 0x7f9b041e4c30 2023-07-06T02:23:48.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.875+0000 7f9b017fa700 1 -- 172.21.15.89:0/2580687393 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9af0058240 con 0x7f9b041e4c30 2023-07-06T02:23:48.878 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:48.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9ae806ad10 msgr2=0x7f9ae806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9ae806ad10 0x7f9ae806d1c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009c60 tx=0x7f9af8009210 comp rx=0 tx=0).stop 2023-07-06T02:23:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041e4c30 msgr2=0x7f9b041f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041e4c30 0x7f9b041f59c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c800 tx=0x7f9af0000fe0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 shutdown_connections 2023-07-06T02:23:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b041f5f00 0x7f9b040c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9af4003fd0 0x7f9b040c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f9ae806ad10 0x7f9ae806d1c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 --2- 172.21.15.89:0/2580687393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041e4c30 0x7f9b041f59c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 >> 172.21.15.89:0/2580687393 conn(0x7f9b040d0b00 msgr2=0x7f9b0406d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 shutdown_connections 2023-07-06T02:23:48.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:48.877+0000 7f9b0b0b6700 1 -- 172.21.15.89:0/2580687393 wait complete. 2023-07-06T02:23:48.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.013+0000 7f0180bc4700 1 Processor -- start 2023-07-06T02:23:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.013+0000 7f0180bc4700 1 -- start start 2023-07-06T02:23:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f0180bc4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f017c0b1ba0 0x7f017c167340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f0180bc4700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017c0cb3d0 con 0x7f017c1600b0 2023-07-06T02:23:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f0180bc4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017c156e80 con 0x7f017c156ab0 2023-07-06T02:23:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f0180bc4700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017c0ca370 con 0x7f017c0b1ba0 2023-07-06T02:23:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f017ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f017c0b1ba0 0x7f017c167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f0179d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f017c1600b0 0x7f017c0b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58882/0 (socket says 172.21.15.89:58882) 2023-07-06T02:23:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f0179d9b700 1 -- 172.21.15.89:0/1449137248 learned_addr learned my addr 172.21.15.89:0/1449137248 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.014+0000 7f017ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f017c0b1ba0 0x7f017c167340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33982/0 (socket says 172.21.15.89:33982) 2023-07-06T02:23:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017ad9d700 1 -- 172.21.15.89:0/1449137248 >> v1:172.21.15.172:6789/0 conn(0x7f017c156ab0 legacy=0x7f017c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017ad9d700 1 -- 172.21.15.89:0/1449137248 >> v1:172.21.15.89:6789/0 conn(0x7f017c1600b0 legacy=0x7f017c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017959a700 1 -- 172.21.15.89:0/1449137248 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486221565 0 0) 0x7f017c0cb3d0 con 0x7f017c1600b0 2023-07-06T02:23:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017ad9d700 1 -- 172.21.15.89:0/1449137248 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c160d80 con 0x7f017c0b1ba0 2023-07-06T02:23:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017ad9d700 1 --2- 172.21.15.89:0/1449137248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c0b1ba0 0x7f017c167340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0174009d90 tx=0x7f017400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57133df0047767a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017959a700 1 -- 172.21.15.89:0/1449137248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400e070 con 0x7f017c0b1ba0 2023-07-06T02:23:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017959a700 1 -- 172.21.15.89:0/1449137248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017400bb90 con 0x7f017c0b1ba0 2023-07-06T02:23:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f017959a700 1 -- 172.21.15.89:0/1449137248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174003a20 con 0x7f017c0b1ba0 2023-07-06T02:23:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f0180bc4700 1 -- 172.21.15.89:0/1449137248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c0b1ba0 msgr2=0x7f017c167340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.015+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1449137248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c0b1ba0 0x7f017c167340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0174009d90 tx=0x7f017400b330 comp rx=0 tx=0).stop 2023-07-06T02:23:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.016+0000 7f0180bc4700 1 -- 172.21.15.89:0/1449137248 shutdown_connections 2023-07-06T02:23:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.016+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1449137248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c0b1ba0 0x7f017c167340 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.016+0000 7f0180bc4700 1 -- 172.21.15.89:0/1449137248 >> 172.21.15.89:0/1449137248 conn(0x7f017c0cc490 msgr2=0x7f017c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.016+0000 7f0180bc4700 1 -- 172.21.15.89:0/1449137248 shutdown_connections 2023-07-06T02:23:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.016+0000 7f0180bc4700 1 -- 172.21.15.89:0/1449137248 wait complete. 2023-07-06T02:23:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.017+0000 7f0180bc4700 1 Processor -- start 2023-07-06T02:23:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.017+0000 7f0180bc4700 1 -- start start 2023-07-06T02:23:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c0b1ba0 0x7f017c1f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c156ab0 0x7f017c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c1600b0 0x7f017c0b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c0b3850 con 0x7f017c156ab0 2023-07-06T02:23:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c1f53f0 con 0x7f017c1600b0 2023-07-06T02:23:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017c0b3a80 con 0x7f017c0b1ba0 2023-07-06T02:23:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c156ab0 0x7f017c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c156ab0 0x7f017c1f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33986/0 (socket says 172.21.15.89:33986) 2023-07-06T02:23:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 -- 172.21.15.89:0/1435921655 learned_addr learned my addr 172.21.15.89:0/1435921655 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f017ad9d700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c1600b0 0x7f017c0b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f017a59c700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c0b1ba0 0x7f017c1f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 -- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c0b1ba0 msgr2=0x7f017c1f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c0b1ba0 0x7f017c1f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 -- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c1600b0 msgr2=0x7f017c0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c1600b0 0x7f017c0b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 -- 172.21.15.89:0/1435921655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0174009a70 con 0x7f017c156ab0 2023-07-06T02:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0179d9b700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c156ab0 0x7f017c1f5e40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f016c00c550 tx=0x7f016c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f016affd700 1 -- 172.21.15.89:0/1435921655 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c004c80 con 0x7f017c156ab0 2023-07-06T02:23:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f016affd700 1 -- 172.21.15.89:0/1435921655 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016c004e00 con 0x7f017c156ab0 2023-07-06T02:23:49.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c0b3d60 con 0x7f017c156ab0 2023-07-06T02:23:49.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f016affd700 1 -- 172.21.15.89:0/1435921655 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c007310 con 0x7f017c156ab0 2023-07-06T02:23:49.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.018+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c1f5920 con 0x7f017c156ab0 2023-07-06T02:23:49.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.019+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017c1f5b40 con 0x7f017c156ab0 2023-07-06T02:23:49.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.020+0000 7f016affd700 1 -- 172.21.15.89:0/1435921655 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f016c007470 con 0x7f017c156ab0 2023-07-06T02:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.020+0000 7f016affd700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f015c06ace0 0x7f015c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.020+0000 7f016affd700 1 -- 172.21.15.89:0/1435921655 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f016c089300 con 0x7f017c156ab0 2023-07-06T02:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.020+0000 7f017a59c700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f015c06ace0 0x7f015c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.021+0000 7f017a59c700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f015c06ace0 0x7f015c06d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0170007fd0 tx=0x7f0170007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.024+0000 7f016affd700 1 -- 172.21.15.89:0/1435921655 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f016c054110 con 0x7f017c156ab0 2023-07-06T02:23:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.211+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f017c074270 con 0x7f017c156ab0 2023-07-06T02:23:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.211+0000 7f016affd700 1 -- 172.21.15.89:0/1435921655 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f016c057730 con 0x7f017c156ab0 2023-07-06T02:23:49.213 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f015c06ace0 msgr2=0x7f015c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f015c06ace0 0x7f015c06d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0170007fd0 tx=0x7f0170007ec0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c156ab0 msgr2=0x7f017c1f5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c156ab0 0x7f017c1f5e40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f016c00c550 tx=0x7f016c00c860 comp rx=0 tx=0).stop 2023-07-06T02:23:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 shutdown_connections 2023-07-06T02:23:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c1600b0 0x7f017c0b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c0b1ba0 0x7f017c1f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f015c06ace0 0x7f015c06d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 --2- 172.21.15.89:0/1435921655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f017c156ab0 0x7f017c1f5e40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 >> 172.21.15.89:0/1435921655 conn(0x7f017c0cc490 msgr2=0x7f017c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 shutdown_connections 2023-07-06T02:23:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.213+0000 7f0180bc4700 1 -- 172.21.15.89:0/1435921655 wait complete. 2023-07-06T02:23:49.227 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.348+0000 7fb766b19700 1 Processor -- start 2023-07-06T02:23:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.348+0000 7fb766b19700 1 -- start start 2023-07-06T02:23:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb766b19700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7600705a0 0x7fb7600ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb766b19700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7600cd180 0x7fb76015a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb766b19700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76006e110 con 0x7fb76006dc10 2023-07-06T02:23:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb766b19700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76006e3f0 con 0x7fb7600705a0 2023-07-06T02:23:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb766b19700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76006e6d0 con 0x7fb7600cd180 2023-07-06T02:23:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb7648b5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7600705a0 0x7fb7600ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb7650b6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb76006dc10 0x7fb760157950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58898/0 (socket says 172.21.15.89:58898) 2023-07-06T02:23:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb7650b6700 1 -- 172.21.15.89:0/774430704 learned_addr learned my addr 172.21.15.89:0/774430704 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb7648b5700 1 -- 172.21.15.89:0/774430704 >> v2:172.21.15.172:3300/0 conn(0x7fb7600cd180 msgr2=0x7fb76015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb7648b5700 1 --2- 172.21.15.89:0/774430704 >> v2:172.21.15.172:3300/0 conn(0x7fb7600cd180 0x7fb76015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb7648b5700 1 -- 172.21.15.89:0/774430704 >> v1:172.21.15.89:6789/0 conn(0x7fb76006dc10 legacy=0x7fb760157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb7648b5700 1 -- 172.21.15.89:0/774430704 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76006e9b0 con 0x7fb7600705a0 2023-07-06T02:23:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.349+0000 7fb75f7fe700 1 -- 172.21.15.89:0/774430704 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352833720 0 0) 0x7fb76006e110 con 0x7fb76006dc10 2023-07-06T02:23:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb7648b5700 1 --2- 172.21.15.89:0/774430704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600705a0 0x7fb7600ccc40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb7600cc4d0 tx=0x7fb754009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954caa8963700cef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb75f7fe700 1 -- 172.21.15.89:0/774430704 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754011830 con 0x7fb7600705a0 2023-07-06T02:23:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb75f7fe700 1 -- 172.21.15.89:0/774430704 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754011990 con 0x7fb7600705a0 2023-07-06T02:23:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb75f7fe700 1 -- 172.21.15.89:0/774430704 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754011d20 con 0x7fb7600705a0 2023-07-06T02:23:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 -- 172.21.15.89:0/774430704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600705a0 msgr2=0x7fb7600ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 --2- 172.21.15.89:0/774430704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600705a0 0x7fb7600ccc40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb7600cc4d0 tx=0x7fb754009b20 comp rx=0 tx=0).stop 2023-07-06T02:23:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 -- 172.21.15.89:0/774430704 shutdown_connections 2023-07-06T02:23:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 --2- 172.21.15.89:0/774430704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600705a0 0x7fb7600ccc40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 --2- 172.21.15.89:0/774430704 >> v2:172.21.15.172:3300/0 conn(0x7fb7600cd180 0x7fb76015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 -- 172.21.15.89:0/774430704 >> 172.21.15.89:0/774430704 conn(0x7fb7600d0b00 msgr2=0x7fb7600d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 -- 172.21.15.89:0/774430704 shutdown_connections 2023-07-06T02:23:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.350+0000 7fb766b19700 1 -- 172.21.15.89:0/774430704 wait complete. 2023-07-06T02:23:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.352+0000 7fb766b19700 1 Processor -- start 2023-07-06T02:23:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.352+0000 7fb766b19700 1 -- start start 2023-07-06T02:23:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb766b19700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb76006dc10 0x7fb7600c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb766b19700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7600705a0 0x7fb76014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb766b19700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600cd180 0x7fb7600c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb766b19700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76014be10 con 0x7fb7600cd180 2023-07-06T02:23:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb766b19700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7600c9b50 con 0x7fb76006dc10 2023-07-06T02:23:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb766b19700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7600c9e30 con 0x7fb7600705a0 2023-07-06T02:23:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600cd180 0x7fb7600c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600cd180 0x7fb7600c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34002/0 (socket says 172.21.15.89:34002) 2023-07-06T02:23:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 -- 172.21.15.89:0/75531079 learned_addr learned my addr 172.21.15.89:0/75531079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 -- 172.21.15.89:0/75531079 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7600705a0 msgr2=0x7fb76014b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7600705a0 0x7fb76014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 -- 172.21.15.89:0/75531079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb76006dc10 msgr2=0x7fb7600c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb76006dc10 0x7fb7600c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 -- 172.21.15.89:0/75531079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7540094a0 con 0x7fb7600cd180 2023-07-06T02:23:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb7650b6700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600cd180 0x7fb7600c5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb75800cba0 tx=0x7fb75800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.353+0000 7fb75d7fa700 1 -- 172.21.15.89:0/75531079 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800ab80 con 0x7fb7600cd180 2023-07-06T02:23:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.354+0000 7fb75d7fa700 1 -- 172.21.15.89:0/75531079 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75800ad00 con 0x7fb7600cd180 2023-07-06T02:23:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.354+0000 7fb75d7fa700 1 -- 172.21.15.89:0/75531079 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758007310 con 0x7fb7600cd180 2023-07-06T02:23:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.354+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7600ca110 con 0x7fb7600cd180 2023-07-06T02:23:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.354+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7600c7490 con 0x7fb7600cd180 2023-07-06T02:23:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.354+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb760066fb0 con 0x7fb7600cd180 2023-07-06T02:23:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.355+0000 7fb75d7fa700 1 -- 172.21.15.89:0/75531079 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb758007470 con 0x7fb7600cd180 2023-07-06T02:23:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.355+0000 7fb75d7fa700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb74406ace0 0x7fb74406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.355+0000 7fb7648b5700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb74406ace0 0x7fb74406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.355+0000 7fb75d7fa700 1 -- 172.21.15.89:0/75531079 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb7580891e0 con 0x7fb7600cd180 2023-07-06T02:23:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.356+0000 7fb7648b5700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb74406ace0 0x7fb74406d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb754000c00 tx=0x7fb754010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.360+0000 7fb75d7fa700 1 -- 172.21.15.89:0/75531079 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb758053ff0 con 0x7fb7600cd180 2023-07-06T02:23:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.551+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb760074270 con 0x7fb7600cd180 2023-07-06T02:23:49.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.551+0000 7fb75d7fa700 1 -- 172.21.15.89:0/75531079 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb758057610 con 0x7fb7600cd180 2023-07-06T02:23:49.554 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb74406ace0 msgr2=0x7fb74406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb74406ace0 0x7fb74406d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb754000c00 tx=0x7fb754010000 comp rx=0 tx=0).stop 2023-07-06T02:23:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600cd180 msgr2=0x7fb7600c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600cd180 0x7fb7600c5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb75800cba0 tx=0x7fb75800cf60 comp rx=0 tx=0).stop 2023-07-06T02:23:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 shutdown_connections 2023-07-06T02:23:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb74406ace0 0x7fb74406d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7600cd180 0x7fb7600c5040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb76006dc10 0x7fb7600c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 --2- 172.21.15.89:0/75531079 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7600705a0 0x7fb76014b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.553+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 >> 172.21.15.89:0/75531079 conn(0x7fb7600d0b00 msgr2=0x7fb760158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:49.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.554+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 shutdown_connections 2023-07-06T02:23:49.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.554+0000 7fb766b19700 1 -- 172.21.15.89:0/75531079 wait complete. 2023-07-06T02:23:49.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:49.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.689+0000 7f72f8148700 1 Processor -- start 2023-07-06T02:23:49.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.690+0000 7f72f8148700 1 -- start start 2023-07-06T02:23:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.690+0000 7f72f8148700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72f006e170 0x7f72f015ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.690+0000 7f72f8148700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f015d390 con 0x7f72f0158960 2023-07-06T02:23:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.690+0000 7f72f8148700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f006c770 con 0x7f72f00ccea0 2023-07-06T02:23:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.690+0000 7f72f8148700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f006d5d0 con 0x7f72f006e170 2023-07-06T02:23:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f66e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72f006e170 0x7f72f015ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f66e5700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f72f006e170 0x7f72f015ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34010/0 (socket says 172.21.15.89:34010) 2023-07-06T02:23:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f66e5700 1 -- 172.21.15.89:0/229330424 learned_addr learned my addr 172.21.15.89:0/229330424 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f56e3700 1 --1- 172.21.15.89:0/229330424 >> v1:172.21.15.89:6789/0 conn(0x7f72f0158960 0x7f72f006da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58908/0 (socket says 172.21.15.89:58908) 2023-07-06T02:23:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f66e5700 1 -- 172.21.15.89:0/229330424 >> v1:172.21.15.172:6789/0 conn(0x7f72f00ccea0 legacy=0x7f72f00c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f66e5700 1 -- 172.21.15.89:0/229330424 >> v1:172.21.15.89:6789/0 conn(0x7f72f0158960 legacy=0x7f72f006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f66e5700 1 -- 172.21.15.89:0/229330424 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f0158d90 con 0x7f72f006e170 2023-07-06T02:23:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f4ee2700 1 -- 172.21.15.89:0/229330424 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244117563 0 0) 0x7f72ec001110 con 0x7f72f0158960 2023-07-06T02:23:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f66e5700 1 --2- 172.21.15.89:0/229330424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f006e170 0x7f72f015ce50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009e50 tx=0x7f72e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4676f12d3bcd0ce2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f4ee2700 1 -- 172.21.15.89:0/229330424 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e800e070 con 0x7f72f006e170 2023-07-06T02:23:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f4ee2700 1 -- 172.21.15.89:0/229330424 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e8003a20 con 0x7f72f006e170 2023-07-06T02:23:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.691+0000 7f72f4ee2700 1 -- 172.21.15.89:0/229330424 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8003dd0 con 0x7f72f006e170 2023-07-06T02:23:49.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.692+0000 7f72f8148700 1 -- 172.21.15.89:0/229330424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f006e170 msgr2=0x7f72f015ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.692+0000 7f72f8148700 1 --2- 172.21.15.89:0/229330424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f006e170 0x7f72f015ce50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009e50 tx=0x7f72e800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.692+0000 7f72f8148700 1 -- 172.21.15.89:0/229330424 shutdown_connections 2023-07-06T02:23:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.692+0000 7f72f8148700 1 --2- 172.21.15.89:0/229330424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f006e170 0x7f72f015ce50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.692+0000 7f72f8148700 1 -- 172.21.15.89:0/229330424 >> 172.21.15.89:0/229330424 conn(0x7f72f00ce870 msgr2=0x7f72f00cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:49.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.692+0000 7f72f8148700 1 -- 172.21.15.89:0/229330424 shutdown_connections 2023-07-06T02:23:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.692+0000 7f72f8148700 1 -- 172.21.15.89:0/229330424 wait complete. 2023-07-06T02:23:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.693+0000 7f72f8148700 1 Processor -- start 2023-07-06T02:23:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.693+0000 7f72f8148700 1 -- start start 2023-07-06T02:23:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f8148700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72f006e170 0x7f72f014a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f8148700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72f00ccea0 0x7f72f015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f8148700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f0158960 0x7f72f0146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f8148700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f014a590 con 0x7f72f0158960 2023-07-06T02:23:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f8148700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f015c0d0 con 0x7f72f006e170 2023-07-06T02:23:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f8148700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f015b2d0 con 0x7f72f00ccea0 2023-07-06T02:23:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f66e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f0158960 0x7f72f0146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f66e5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f0158960 0x7f72f0146c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34016/0 (socket says 172.21.15.89:34016) 2023-07-06T02:23:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.694+0000 7f72f66e5700 1 -- 172.21.15.89:0/868513122 learned_addr learned my addr 172.21.15.89:0/868513122 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f5ee4700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72f006e170 0x7f72f014a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f56e3700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72f00ccea0 0x7f72f015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f66e5700 1 -- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72f00ccea0 msgr2=0x7f72f015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f66e5700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72f00ccea0 0x7f72f015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f66e5700 1 -- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72f006e170 msgr2=0x7f72f014a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f66e5700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72f006e170 0x7f72f014a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f66e5700 1 -- 172.21.15.89:0/868513122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8009b30 con 0x7f72f0158960 2023-07-06T02:23:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f5ee4700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72f006e170 0x7f72f014a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f66e5700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f0158960 0x7f72f0146c20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f72f0072840 tx=0x7f72e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72e67fc700 1 -- 172.21.15.89:0/868513122 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8004440 con 0x7f72f0158960 2023-07-06T02:23:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72e67fc700 1 -- 172.21.15.89:0/868513122 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e80045a0 con 0x7f72f0158960 2023-07-06T02:23:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72e67fc700 1 -- 172.21.15.89:0/868513122 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8004950 con 0x7f72f0158960 2023-07-06T02:23:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f015b550 con 0x7f72f0158960 2023-07-06T02:23:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.695+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f015bb10 con 0x7f72f0158960 2023-07-06T02:23:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.696+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f01475d0 con 0x7f72f0158960 2023-07-06T02:23:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.696+0000 7f72e67fc700 1 -- 172.21.15.89:0/868513122 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72e8025450 con 0x7f72f0158960 2023-07-06T02:23:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.697+0000 7f72e67fc700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f72d406ace0 0x7f72d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.697+0000 7f72f5ee4700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f72d406ace0 0x7f72d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.697+0000 7f72e67fc700 1 -- 172.21.15.89:0/868513122 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f72e808aff0 con 0x7f72f0158960 2023-07-06T02:23:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.697+0000 7f72f5ee4700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f72d406ace0 0x7f72d406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72e0003fc0 tx=0x7f72e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.702+0000 7f72e67fc700 1 -- 172.21.15.89:0/868513122 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f72f01475d0 con 0x7f72f0158960 2023-07-06T02:23:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.888+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72f01475d0 con 0x7f72f0158960 2023-07-06T02:23:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.888+0000 7f72e67fc700 1 -- 172.21.15.89:0/868513122 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72f01475d0 con 0x7f72f0158960 2023-07-06T02:23:49.890 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f72d406ace0 msgr2=0x7f72d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f72d406ace0 0x7f72d406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72e0003fc0 tx=0x7f72e0009040 comp rx=0 tx=0).stop 2023-07-06T02:23:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f0158960 msgr2=0x7f72f0146c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f0158960 0x7f72f0146c20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f72f0072840 tx=0x7f72e8004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 shutdown_connections 2023-07-06T02:23:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f72d406ace0 0x7f72d406d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72f0158960 0x7f72f0146c20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72f006e170 0x7f72f014a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 --2- 172.21.15.89:0/868513122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72f00ccea0 0x7f72f015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 >> 172.21.15.89:0/868513122 conn(0x7f72f00ce870 msgr2=0x7f72f00cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 shutdown_connections 2023-07-06T02:23:49.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:49.890+0000 7f72f8148700 1 -- 172.21.15.89:0/868513122 wait complete. 2023-07-06T02:23:49.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:50.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.027+0000 7f5c30755700 1 Processor -- start 2023-07-06T02:23:50.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.027+0000 7f5c30755700 1 -- start start 2023-07-06T02:23:50.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c30755700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5c2806c6b0 0x7f5c280b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c30755700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280cab50 con 0x7f5c2815c690 2023-07-06T02:23:50.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c30755700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2815db70 con 0x7f5c280b1490 2023-07-06T02:23:50.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c30755700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2815de50 con 0x7f5c2806c6b0 2023-07-06T02:23:50.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2e4f1700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5c2815c690 0x7f5c280cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58922/0 (socket says 172.21.15.89:58922) 2023-07-06T02:23:50.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2e4f1700 1 -- 172.21.15.89:0/4289048154 learned_addr learned my addr 172.21.15.89:0/4289048154 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:50.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2ecf2700 1 --2- 172.21.15.89:0/4289048154 >> v2:172.21.15.172:3300/0 conn(0x7f5c2806c6b0 0x7f5c280b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182142499 0 0) 0x7f5c280cab50 con 0x7f5c2815c690 2023-07-06T02:23:50.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c140035b0 con 0x7f5c2815c690 2023-07-06T02:23:50.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442321489 0 0) 0x7f5c2815db70 con 0x7f5c280b1490 2023-07-06T02:23:50.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c280cab50 con 0x7f5c280b1490 2023-07-06T02:23:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24579371 0 0) 0x7f5c140035b0 con 0x7f5c2815c690 2023-07-06T02:23:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 >> v2:172.21.15.172:3300/0 conn(0x7f5c2806c6b0 msgr2=0x7f5c280b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 --2- 172.21.15.89:0/4289048154 >> v2:172.21.15.172:3300/0 conn(0x7f5c2806c6b0 0x7f5c280b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 >> v1:172.21.15.172:6789/0 conn(0x7f5c280b1490 legacy=0x7f5c2806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280cad80 con 0x7f5c2815c690 2023-07-06T02:23:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f5c24001670 con 0x7f5c2815c690 2023-07-06T02:23:50.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 >> v1:172.21.15.89:6789/0 conn(0x7f5c2815c690 legacy=0x7f5c280cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c14003be0 0x7f5c14003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14004850 0x7f5c14004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c14005200 0x7f5c1400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2815de50 con 0x7f5c14005200 2023-07-06T02:23:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1400c0e0 con 0x7f5c14004850 2023-07-06T02:23:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.028+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1400c270 con 0x7f5c14003be0 2023-07-06T02:23:50.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c14005200 0x7f5c1400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2e4f1700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c14003be0 0x7f5c14003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2dcf0700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14004850 0x7f5c14004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 -- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c14003be0 msgr2=0x7f5c14003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:50.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c14003be0 0x7f5c14003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 -- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14004850 msgr2=0x7f5c14004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:50.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14004850 0x7f5c14004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 -- 172.21.15.89:0/4289048154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140035b0 con 0x7f5c14005200 2023-07-06T02:23:50.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 -- 172.21.15.89:0/4289048154 reap_dead start 2023-07-06T02:23:50.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2ecf2700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c14005200 0x7f5c1400ba10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009f90 tx=0x7f5c14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a58ff5b486716a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20003a20 con 0x7f5c14005200 2023-07-06T02:23:50.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2000f040 con 0x7f5c14005200 2023-07-06T02:23:50.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.029+0000 7f5c2d4ef700 1 -- 172.21.15.89:0/4289048154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20003dc0 con 0x7f5c14005200 2023-07-06T02:23:50.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.030+0000 7f5c30755700 1 -- 172.21.15.89:0/4289048154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c14005200 msgr2=0x7f5c1400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.030+0000 7f5c30755700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c14005200 0x7f5c1400ba10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009f90 tx=0x7f5c14005880 comp rx=0 tx=0).stop 2023-07-06T02:23:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.030+0000 7f5c30755700 1 -- 172.21.15.89:0/4289048154 shutdown_connections 2023-07-06T02:23:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.030+0000 7f5c30755700 1 --2- 172.21.15.89:0/4289048154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c14005200 0x7f5c1400ba10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.030+0000 7f5c30755700 1 -- 172.21.15.89:0/4289048154 >> 172.21.15.89:0/4289048154 conn(0x7f5c280cc490 msgr2=0x7f5c280cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.030+0000 7f5c30755700 1 -- 172.21.15.89:0/4289048154 shutdown_connections 2023-07-06T02:23:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.030+0000 7f5c30755700 1 -- 172.21.15.89:0/4289048154 wait complete. 2023-07-06T02:23:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.031+0000 7f5c30755700 1 Processor -- start 2023-07-06T02:23:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.031+0000 7f5c30755700 1 -- start start 2023-07-06T02:23:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14005200 0x7f5c280b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28154ab0 0x7f5c280b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c280b0e60 0x7f5c280b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280b1300 con 0x7f5c28154ab0 2023-07-06T02:23:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28158760 con 0x7f5c14005200 2023-07-06T02:23:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280b44b0 con 0x7f5c280b0e60 2023-07-06T02:23:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28154ab0 0x7f5c280b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28154ab0 0x7f5c280b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34034/0 (socket says 172.21.15.89:34034) 2023-07-06T02:23:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 -- 172.21.15.89:0/3272656 learned_addr learned my addr 172.21.15.89:0/3272656 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2ecf2700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c280b0e60 0x7f5c280b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2e4f1700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14005200 0x7f5c280b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 -- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c280b0e60 msgr2=0x7f5c280b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c280b0e60 0x7f5c280b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 -- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14005200 msgr2=0x7f5c280b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14005200 0x7f5c280b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 -- 172.21.15.89:0/3272656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280b4790 con 0x7f5c28154ab0 2023-07-06T02:23:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2ecf2700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c280b0e60 0x7f5c280b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c2dcf0700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28154ab0 0x7f5c280b0920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800f810 tx=0x7f5c1800fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c1effd700 1 -- 172.21.15.89:0/3272656 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18005320 con 0x7f5c28154ab0 2023-07-06T02:23:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c1effd700 1 -- 172.21.15.89:0/3272656 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c180054a0 con 0x7f5c28154ab0 2023-07-06T02:23:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c280b4a70 con 0x7f5c28154ab0 2023-07-06T02:23:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c1effd700 1 -- 172.21.15.89:0/3272656 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18006aa0 con 0x7f5c28154ab0 2023-07-06T02:23:50.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.032+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c280b5030 con 0x7f5c28154ab0 2023-07-06T02:23:50.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.033+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c280b5250 con 0x7f5c28154ab0 2023-07-06T02:23:50.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.034+0000 7f5c1effd700 1 -- 172.21.15.89:0/3272656 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c18006c00 con 0x7f5c28154ab0 2023-07-06T02:23:50.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.034+0000 7f5c1effd700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5c0c06ace0 0x7f5c0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.034+0000 7f5c1effd700 1 -- 172.21.15.89:0/3272656 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c18089120 con 0x7f5c28154ab0 2023-07-06T02:23:50.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.034+0000 7f5c2e4f1700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5c0c06ace0 0x7f5c0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.035+0000 7f5c2e4f1700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5c0c06ace0 0x7f5c0c06d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c24001050 tx=0x7f5c24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.039+0000 7f5c1effd700 1 -- 172.21.15.89:0/3272656 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5c18053ea0 con 0x7f5c28154ab0 2023-07-06T02:23:50.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.227+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5c28074270 con 0x7f5c28154ab0 2023-07-06T02:23:50.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.227+0000 7f5c1effd700 1 -- 172.21.15.89:0/3272656 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5c180574c0 con 0x7f5c28154ab0 2023-07-06T02:23:50.230 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:50.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.229+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5c0c06ace0 msgr2=0x7f5c0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.229+0000 7f5c30755700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5c0c06ace0 0x7f5c0c06d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c24001050 tx=0x7f5c24009040 comp rx=0 tx=0).stop 2023-07-06T02:23:50.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.229+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28154ab0 msgr2=0x7f5c280b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.229+0000 7f5c30755700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28154ab0 0x7f5c280b0920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800f810 tx=0x7f5c1800fbd0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 shutdown_connections 2023-07-06T02:23:50.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c280b0e60 0x7f5c280b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c14005200 0x7f5c280b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5c0c06ace0 0x7f5c0c06d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 --2- 172.21.15.89:0/3272656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28154ab0 0x7f5c280b0920 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 >> 172.21.15.89:0/3272656 conn(0x7f5c280cc490 msgr2=0x7f5c2815bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 shutdown_connections 2023-07-06T02:23:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.230+0000 7f5c30755700 1 -- 172.21.15.89:0/3272656 wait complete. 2023-07-06T02:23:50.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T02:23:50.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.364+0000 7f68a80b6700 1 Processor -- start 2023-07-06T02:23:50.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.364+0000 7f68a80b6700 1 -- start start 2023-07-06T02:23:50.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a80b6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f68a01600b0 0x7f68a00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a80b6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a00cb3d0 con 0x7f68a0156ab0 2023-07-06T02:23:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a80b6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a00b1dd0 con 0x7f68a00b19d0 2023-07-06T02:23:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a80b6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a0156e80 con 0x7f68a01600b0 2023-07-06T02:23:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a5e52700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f68a0156ab0 0x7f68a0160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58942/0 (socket says 172.21.15.89:58942) 2023-07-06T02:23:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a5e52700 1 -- 172.21.15.89:0/946496693 learned_addr learned my addr 172.21.15.89:0/946496693 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222544423 0 0) 0x7f68a00cb3d0 con 0x7f68a0156ab0 2023-07-06T02:23:50.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f688c0035b0 con 0x7f68a0156ab0 2023-07-06T02:23:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.365+0000 7f68a5651700 1 --2- 172.21.15.89:0/946496693 >> v2:172.21.15.172:3300/0 conn(0x7f68a01600b0 0x7f68a00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2298098750 0 0) 0x7f688c0035b0 con 0x7f68a0156ab0 2023-07-06T02:23:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 >> v2:172.21.15.172:3300/0 conn(0x7f68a01600b0 msgr2=0x7f68a00b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 --2- 172.21.15.89:0/946496693 >> v2:172.21.15.172:3300/0 conn(0x7f68a01600b0 0x7f68a00b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 >> v1:172.21.15.172:6789/0 conn(0x7f68a00b19d0 legacy=0x7f68a00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00ca370 con 0x7f68a0156ab0 2023-07-06T02:23:50.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f6890001670 con 0x7f68a0156ab0 2023-07-06T02:23:50.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 >> v1:172.21.15.89:6789/0 conn(0x7f68a0156ab0 legacy=0x7f68a0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f688c003a00 0x7f688c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004850 0x7f688c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c005200 0x7f688c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0156e80 con 0x7f688c003a00 2023-07-06T02:23:50.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c080 con 0x7f688c004850 2023-07-06T02:23:50.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c240 con 0x7f688c005200 2023-07-06T02:23:50.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5e52700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f688c003a00 0x7f688c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5651700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004850 0x7f688c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a6653700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c005200 0x7f688c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5e52700 1 -- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c005200 msgr2=0x7f688c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:50.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5e52700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c005200 0x7f688c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5e52700 1 -- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004850 msgr2=0x7f688c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:50.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5e52700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004850 0x7f688c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5e52700 1 -- 172.21.15.89:0/946496693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c0035b0 con 0x7f688c003a00 2023-07-06T02:23:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a6653700 1 -- 172.21.15.89:0/946496693 reap_dead start 2023-07-06T02:23:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a5e52700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f688c003a00 0x7f688c003e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f689000c2a0 tx=0x7f688c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5e6047fc917b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890007c10 con 0x7f688c003a00 2023-07-06T02:23:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689000cdc0 con 0x7f688c003a00 2023-07-06T02:23:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.366+0000 7f68a4e50700 1 -- 172.21.15.89:0/946496693 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890005ba0 con 0x7f688c003a00 2023-07-06T02:23:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.367+0000 7f68a80b6700 1 -- 172.21.15.89:0/946496693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f688c003a00 msgr2=0x7f688c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.367+0000 7f68a80b6700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f688c003a00 0x7f688c003e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f689000c2a0 tx=0x7f688c005880 comp rx=0 tx=0).stop 2023-07-06T02:23:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.367+0000 7f68a80b6700 1 -- 172.21.15.89:0/946496693 shutdown_connections 2023-07-06T02:23:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.367+0000 7f68a80b6700 1 --2- 172.21.15.89:0/946496693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f688c003a00 0x7f688c003e10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.367+0000 7f68a80b6700 1 -- 172.21.15.89:0/946496693 >> 172.21.15.89:0/946496693 conn(0x7f68a00cc490 msgr2=0x7f68a00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.367+0000 7f68a80b6700 1 -- 172.21.15.89:0/946496693 shutdown_connections 2023-07-06T02:23:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.367+0000 7f68a80b6700 1 -- 172.21.15.89:0/946496693 wait complete. 2023-07-06T02:23:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.368+0000 7f68a80b6700 1 Processor -- start 2023-07-06T02:23:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.368+0000 7f68a80b6700 1 -- start start 2023-07-06T02:23:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a80b6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c003a00 0x7f68a0153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a80b6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68a00c3540 0x7f68a00ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a80b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68a00abe30 0x7f68a00b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a80b6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00b02c0 con 0x7f68a00abe30 2023-07-06T02:23:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a80b6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00af9f0 con 0x7f688c003a00 2023-07-06T02:23:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a80b6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00c3a40 con 0x7f68a00c3540 2023-07-06T02:23:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68a00abe30 0x7f68a00b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68a00abe30 0x7f68a00b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34056/0 (socket says 172.21.15.89:34056) 2023-07-06T02:23:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 -- 172.21.15.89:0/2490195853 learned_addr learned my addr 172.21.15.89:0/2490195853 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 -- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68a00c3540 msgr2=0x7f68a00ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a5651700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68a00c3540 0x7f68a00ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a5e52700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c003a00 0x7f68a0153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68a00c3540 0x7f68a00ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 -- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c003a00 msgr2=0x7f68a0153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c003a00 0x7f68a0153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.369+0000 7f68a6653700 1 -- 172.21.15.89:0/2490195853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00afc00 con 0x7f68a00abe30 2023-07-06T02:23:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.370+0000 7f68a6653700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68a00abe30 0x7f68a00b06e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f68a006d6c0 tx=0x7f689800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.370+0000 7f68967fc700 1 -- 172.21.15.89:0/2490195853 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898003c10 con 0x7f68a00abe30 2023-07-06T02:23:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.370+0000 7f68967fc700 1 -- 172.21.15.89:0/2490195853 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898003d70 con 0x7f68a00abe30 2023-07-06T02:23:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.370+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a00afee0 con 0x7f68a00abe30 2023-07-06T02:23:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.370+0000 7f68967fc700 1 -- 172.21.15.89:0/2490195853 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898005310 con 0x7f68a00abe30 2023-07-06T02:23:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.370+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a01542d0 con 0x7f68a00abe30 2023-07-06T02:23:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.371+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a01544f0 con 0x7f68a00abe30 2023-07-06T02:23:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.371+0000 7f68967fc700 1 -- 172.21.15.89:0/2490195853 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6898005470 con 0x7f68a00abe30 2023-07-06T02:23:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.372+0000 7f68967fc700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f688406ace0 0x7f688406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.372+0000 7f68967fc700 1 -- 172.21.15.89:0/2490195853 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6898089790 con 0x7f68a00abe30 2023-07-06T02:23:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.372+0000 7f68a5e52700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f688406ace0 0x7f688406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.372+0000 7f68a5e52700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f688406ace0 0x7f688406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6890015040 tx=0x7f6890007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.376+0000 7f68967fc700 1 -- 172.21.15.89:0/2490195853 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68a01544f0 con 0x7f68a00abe30 2023-07-06T02:23:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.579+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68a00b0400 con 0x7f68a00abe30 2023-07-06T02:23:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.580+0000 7f68967fc700 1 -- 172.21.15.89:0/2490195853 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f68a00b0400 con 0x7f68a00abe30 2023-07-06T02:23:50.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.581+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f688406ace0 msgr2=0x7f688406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.581+0000 7f68a80b6700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f688406ace0 0x7f688406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6890015040 tx=0x7f6890007000 comp rx=0 tx=0).stop 2023-07-06T02:23:50.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.581+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68a00abe30 msgr2=0x7f68a00b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.581+0000 7f68a80b6700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68a00abe30 0x7f68a00b06e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f68a006d6c0 tx=0x7f689800ee00 comp rx=0 tx=0).stop 2023-07-06T02:23:50.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 shutdown_connections 2023-07-06T02:23:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f688406ace0 0x7f688406d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68a00abe30 0x7f68a00b06e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c003a00 0x7f68a0153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 --2- 172.21.15.89:0/2490195853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68a00c3540 0x7f68a00ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 >> 172.21.15.89:0/2490195853 conn(0x7f68a00cc490 msgr2=0x7f68a0155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 shutdown_connections 2023-07-06T02:23:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.582+0000 7f68a80b6700 1 -- 172.21.15.89:0/2490195853 wait complete. 2023-07-06T02:23:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-07-06T02:23:50.595 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-07-06T02:23:50.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:50.596 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-07-06T02:23:50.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:23:50.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:50.634 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T02:23:50.690 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:50.691 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:50.712 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:50.712 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:50.772 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:23:50.772 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:23:50.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:50.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.949+0000 7fb1a6be6700 1 Processor -- start 2023-07-06T02:23:50.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.949+0000 7fb1a6be6700 1 -- start start 2023-07-06T02:23:50.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a6be6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a006dc10 0x7fb1a015a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a6be6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1a015adb0 0x7fb1a0157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a6be6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a00cd350 con 0x7fb1a00705a0 2023-07-06T02:23:50.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a6be6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a006e110 con 0x7fb1a015adb0 2023-07-06T02:23:50.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a6be6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a006e3f0 con 0x7fb1a006dc10 2023-07-06T02:23:50.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a5183700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1a015adb0 0x7fb1a0157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a4982700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb1a00705a0 0x7fb1a00ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58962/0 (socket says 172.21.15.89:58962) 2023-07-06T02:23:50.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.950+0000 7fb1a4982700 1 -- 172.21.15.89:0/1494027417 learned_addr learned my addr 172.21.15.89:0/1494027417 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:50.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb19ffff700 1 --2- 172.21.15.89:0/1494027417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a006dc10 0x7fb1a015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb1a5183700 1 -- 172.21.15.89:0/1494027417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a006dc10 msgr2=0x7fb1a015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:50.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb1a5183700 1 --2- 172.21.15.89:0/1494027417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a006dc10 0x7fb1a015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb1a5183700 1 -- 172.21.15.89:0/1494027417 >> v1:172.21.15.89:6789/0 conn(0x7fb1a00705a0 legacy=0x7fb1a00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb1a5183700 1 -- 172.21.15.89:0/1494027417 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a006e6d0 con 0x7fb1a015adb0 2023-07-06T02:23:50.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb19f7fe700 1 -- 172.21.15.89:0/1494027417 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428675902 0 0) 0x7fb1a00cd350 con 0x7fb1a00705a0 2023-07-06T02:23:50.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb19ffff700 1 --2- 172.21.15.89:0/1494027417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a006dc10 0x7fb1a015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:50.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb1a5183700 1 --2- 172.21.15.89:0/1494027417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a015adb0 0x7fb1a0157950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb198009d90 tx=0x7fb19800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fc2d1c3bfc0dec server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb19f7fe700 1 -- 172.21.15.89:0/1494027417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800e070 con 0x7fb1a015adb0 2023-07-06T02:23:50.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb19f7fe700 1 -- 172.21.15.89:0/1494027417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19800bbd0 con 0x7fb1a015adb0 2023-07-06T02:23:50.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.951+0000 7fb19f7fe700 1 -- 172.21.15.89:0/1494027417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198012430 con 0x7fb1a015adb0 2023-07-06T02:23:50.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 -- 172.21.15.89:0/1494027417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a015adb0 msgr2=0x7fb1a0157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/1494027417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a015adb0 0x7fb1a0157950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb198009d90 tx=0x7fb19800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:50.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 -- 172.21.15.89:0/1494027417 shutdown_connections 2023-07-06T02:23:50.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/1494027417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a015adb0 0x7fb1a0157950 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/1494027417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a006dc10 0x7fb1a015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 -- 172.21.15.89:0/1494027417 >> 172.21.15.89:0/1494027417 conn(0x7fb1a00d0b00 msgr2=0x7fb1a00d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:50.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 -- 172.21.15.89:0/1494027417 shutdown_connections 2023-07-06T02:23:50.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.952+0000 7fb1a6be6700 1 -- 172.21.15.89:0/1494027417 wait complete. 2023-07-06T02:23:50.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.953+0000 7fb1a6be6700 1 Processor -- start 2023-07-06T02:23:50.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.953+0000 7fb1a6be6700 1 -- start start 2023-07-06T02:23:50.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a6be6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a006dc10 0x7fb1a0156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a6be6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a00705a0 0x7fb1a00c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a6be6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1a015adb0 0x7fb1a00c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a6be6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00a7550 con 0x7fb1a006dc10 2023-07-06T02:23:50.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a6be6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00a7830 con 0x7fb1a015adb0 2023-07-06T02:23:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a6be6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00c4f10 con 0x7fb1a00705a0 2023-07-06T02:23:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a4982700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a006dc10 0x7fb1a0156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a4982700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a006dc10 0x7fb1a0156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34066/0 (socket says 172.21.15.89:34066) 2023-07-06T02:23:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.954+0000 7fb1a4982700 1 -- 172.21.15.89:0/980151120 learned_addr learned my addr 172.21.15.89:0/980151120 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb19ffff700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a00705a0 0x7fb1a00c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a5183700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1a015adb0 0x7fb1a00c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a4982700 1 -- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a00705a0 msgr2=0x7fb1a00c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a4982700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a00705a0 0x7fb1a00c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a4982700 1 -- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1a015adb0 msgr2=0x7fb1a00c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a4982700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1a015adb0 0x7fb1a00c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a4982700 1 -- 172.21.15.89:0/980151120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009a70 con 0x7fb1a006dc10 2023-07-06T02:23:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a5183700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1a015adb0 0x7fb1a00c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a4982700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a006dc10 0x7fb1a0156710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c750 tx=0x7fb18800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb19d7fa700 1 -- 172.21.15.89:0/980151120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188004c80 con 0x7fb1a006dc10 2023-07-06T02:23:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb19d7fa700 1 -- 172.21.15.89:0/980151120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188004e00 con 0x7fb1a006dc10 2023-07-06T02:23:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb19d7fa700 1 -- 172.21.15.89:0/980151120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188007310 con 0x7fb1a006dc10 2023-07-06T02:23:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a00c51f0 con 0x7fb1a006dc10 2023-07-06T02:23:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.955+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a0156c50 con 0x7fb1a006dc10 2023-07-06T02:23:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.956+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a0074270 con 0x7fb1a006dc10 2023-07-06T02:23:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.956+0000 7fb19d7fa700 1 -- 172.21.15.89:0/980151120 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb188007470 con 0x7fb1a006dc10 2023-07-06T02:23:50.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.957+0000 7fb19d7fa700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb18006ace0 0x7fb18006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.957+0000 7fb19d7fa700 1 -- 172.21.15.89:0/980151120 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb188089530 con 0x7fb1a006dc10 2023-07-06T02:23:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.957+0000 7fb19ffff700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb18006ace0 0x7fb18006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.957+0000 7fb19ffff700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb18006ace0 0x7fb18006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00cab70 tx=0x7fb190008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:50.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:50.961+0000 7fb19d7fa700 1 -- 172.21.15.89:0/980151120 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb1880542a0 con 0x7fb1a006dc10 2023-07-06T02:23:51.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.149+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1a00cbf80 con 0x7fb1a006dc10 2023-07-06T02:23:51.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.151+0000 7fb19d7fa700 1 -- 172.21.15.89:0/980151120 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fb1880578c0 con 0x7fb1a006dc10 2023-07-06T02:23:51.153 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:51.153 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:23:41.389931+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb18006ace0 msgr2=0x7fb18006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb18006ace0 0x7fb18006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00cab70 tx=0x7fb190008040 comp rx=0 tx=0).stop 2023-07-06T02:23:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a006dc10 msgr2=0x7fb1a0156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a006dc10 0x7fb1a0156710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c750 tx=0x7fb18800cb10 comp rx=0 tx=0).stop 2023-07-06T02:23:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 shutdown_connections 2023-07-06T02:23:51.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1a015adb0 0x7fb1a00c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb18006ace0 0x7fb18006d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1a006dc10 0x7fb1a0156710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 --2- 172.21.15.89:0/980151120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1a00705a0 0x7fb1a00c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 >> 172.21.15.89:0/980151120 conn(0x7fb1a00d0b00 msgr2=0x7fb1a00ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:51.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 shutdown_connections 2023-07-06T02:23:51.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.153+0000 7fb1a6be6700 1 -- 172.21.15.89:0/980151120 wait complete. 2023-07-06T02:23:51.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:51.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:51.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.291+0000 7f35cb977700 1 Processor -- start 2023-07-06T02:23:51.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35cb977700 1 -- start start 2023-07-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35cb977700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c40cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35cb977700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35c40cbda0 0x7f35c40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35cb977700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c40cab50 con 0x7f35c40b19d0 2023-07-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35cb977700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c415de50 con 0x7f35c40cbda0 2023-07-06T02:23:51.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35cb977700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c415db70 con 0x7f35c415c690 2023-07-06T02:23:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35c8f12700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35c40cbda0 0x7f35c40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.292+0000 7f35c9f14700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f35c40b19d0 0x7f35c406bfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58980/0 (socket says 172.21.15.89:58980) 2023-07-06T02:23:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35c9f14700 1 -- 172.21.15.89:0/138315679 learned_addr learned my addr 172.21.15.89:0/138315679 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35c9713700 1 --2- 172.21.15.89:0/138315679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c40cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35c8f12700 1 -- 172.21.15.89:0/138315679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 msgr2=0x7f35c40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35c8f12700 1 --2- 172.21.15.89:0/138315679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c40cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35c8f12700 1 -- 172.21.15.89:0/138315679 >> v1:172.21.15.89:6789/0 conn(0x7f35c40b19d0 legacy=0x7f35c406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35c8f12700 1 -- 172.21.15.89:0/138315679 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c40b1e30 con 0x7f35c40cbda0 2023-07-06T02:23:51.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35bbfff700 1 -- 172.21.15.89:0/138315679 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666689036 0 0) 0x7f35c40cab50 con 0x7f35c40b19d0 2023-07-06T02:23:51.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35c8f12700 1 --2- 172.21.15.89:0/138315679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40cbda0 0x7f35c40b1490 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009840 tx=0x7f35c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cddea9347537ed server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35bbfff700 1 -- 172.21.15.89:0/138315679 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0011830 con 0x7f35c40cbda0 2023-07-06T02:23:51.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35bbfff700 1 -- 172.21.15.89:0/138315679 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0011990 con 0x7f35c40cbda0 2023-07-06T02:23:51.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.293+0000 7f35bbfff700 1 -- 172.21.15.89:0/138315679 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0011d20 con 0x7f35c40cbda0 2023-07-06T02:23:51.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 -- 172.21.15.89:0/138315679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40cbda0 msgr2=0x7f35c40b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 --2- 172.21.15.89:0/138315679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40cbda0 0x7f35c40b1490 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009840 tx=0x7f35c0009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:51.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 -- 172.21.15.89:0/138315679 shutdown_connections 2023-07-06T02:23:51.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 --2- 172.21.15.89:0/138315679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 --2- 172.21.15.89:0/138315679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40cbda0 0x7f35c40b1490 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 -- 172.21.15.89:0/138315679 >> 172.21.15.89:0/138315679 conn(0x7f35c40cc490 msgr2=0x7f35c40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:51.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 -- 172.21.15.89:0/138315679 shutdown_connections 2023-07-06T02:23:51.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.294+0000 7f35cb977700 1 -- 172.21.15.89:0/138315679 wait complete. 2023-07-06T02:23:51.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.295+0000 7f35cb977700 1 Processor -- start 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.295+0000 7f35cb977700 1 -- start start 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35cb977700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40b19d0 0x7f35c40c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35cb977700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35c40cbda0 0x7f35c40b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35cb977700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c41512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35cb977700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4147990 con 0x7f35c40b19d0 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35cb977700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c40b5700 con 0x7f35c40cbda0 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35cb977700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4150210 con 0x7f35c415c690 2023-07-06T02:23:51.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40b19d0 0x7f35c40c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40b19d0 0x7f35c40c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34080/0 (socket says 172.21.15.89:34080) 2023-07-06T02:23:51.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 -- 172.21.15.89:0/3472363563 learned_addr learned my addr 172.21.15.89:0/3472363563 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:51.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9f14700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c41512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c8f12700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35c40cbda0 0x7f35c40b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 -- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 msgr2=0x7f35c41512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c41512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 -- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35c40cbda0 msgr2=0x7f35c40b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35c40cbda0 0x7f35c40b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 -- 172.21.15.89:0/3472363563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c00094a0 con 0x7f35c40b19d0 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35c9713700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40b19d0 0x7f35c40c8d30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f35b400e810 tx=0x7f35b400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35b9ffb700 1 -- 172.21.15.89:0/3472363563 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4013610 con 0x7f35c40b19d0 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35b9ffb700 1 -- 172.21.15.89:0/3472363563 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b4013770 con 0x7f35c40b19d0 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35b9ffb700 1 -- 172.21.15.89:0/3472363563 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4013b20 con 0x7f35c40b19d0 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.296+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c41504f0 con 0x7f35c40b19d0 2023-07-06T02:23:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.297+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c4150ab0 con 0x7f35c40b19d0 2023-07-06T02:23:51.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.297+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c4074270 con 0x7f35c40b19d0 2023-07-06T02:23:51.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.298+0000 7f35b9ffb700 1 -- 172.21.15.89:0/3472363563 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35b4013c80 con 0x7f35c40b19d0 2023-07-06T02:23:51.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.299+0000 7f35b9ffb700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35a806ace0 0x7f35a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.299+0000 7f35c8f12700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35a806ace0 0x7f35a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.299+0000 7f35b9ffb700 1 -- 172.21.15.89:0/3472363563 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f35b40898d0 con 0x7f35c40b19d0 2023-07-06T02:23:51.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.299+0000 7f35c8f12700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35a806ace0 0x7f35a806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35c00090b0 tx=0x7f35c0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.303+0000 7f35b9ffb700 1 -- 172.21.15.89:0/3472363563 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35b40546e0 con 0x7f35c40b19d0 2023-07-06T02:23:51.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.492+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35c406bd80 con 0x7f35c40b19d0 2023-07-06T02:23:51.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.492+0000 7f35b9ffb700 1 -- 172.21.15.89:0/3472363563 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35b4057d00 con 0x7f35c40b19d0 2023-07-06T02:23:51.494 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:51.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35a806ace0 msgr2=0x7f35a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35a806ace0 0x7f35a806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35c00090b0 tx=0x7f35c0008030 comp rx=0 tx=0).stop 2023-07-06T02:23:51.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40b19d0 msgr2=0x7f35c40c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40b19d0 0x7f35c40c8d30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f35b400e810 tx=0x7f35b400ebd0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 shutdown_connections 2023-07-06T02:23:51.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35c415c690 0x7f35c41512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35a806ace0 0x7f35a806d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35c40b19d0 0x7f35c40c8d30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 --2- 172.21.15.89:0/3472363563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35c40cbda0 0x7f35c40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 >> 172.21.15.89:0/3472363563 conn(0x7f35c40cc490 msgr2=0x7f35c406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:51.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 shutdown_connections 2023-07-06T02:23:51.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.494+0000 7f35cb977700 1 -- 172.21.15.89:0/3472363563 wait complete. 2023-07-06T02:23:51.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.629+0000 7f5f174ee700 1 Processor -- start 2023-07-06T02:23:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.629+0000 7f5f174ee700 1 -- start start 2023-07-06T02:23:51.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f174ee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 0x7f5f10160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f174ee700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5f101600b0 0x7f5f100cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f174ee700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f100cb3d0 con 0x7f5f10160bb0 2023-07-06T02:23:51.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f174ee700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f10156e80 con 0x7f5f101600b0 2023-07-06T02:23:51.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f174ee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100ca370 con 0x7f5f10156ab0 2023-07-06T02:23:51.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f14a89700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5f10160bb0 0x7f5f100b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58990/0 (socket says 172.21.15.89:58990) 2023-07-06T02:23:51.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f14a89700 1 -- 172.21.15.89:0/4082411948 learned_addr learned my addr 172.21.15.89:0/4082411948 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f15a8b700 1 --2- 172.21.15.89:0/4082411948 >> v2:172.21.15.172:3300/0 conn(0x7f5f101600b0 0x7f5f100cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f1528a700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 0x7f5f10160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927690347 0 0) 0x7f5f100cb3d0 con 0x7f5f10160bb0 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5efc0035b0 con 0x7f5f10160bb0 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2027776479 0 0) 0x7f5efc0035b0 con 0x7f5f10160bb0 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 msgr2=0x7f5f10160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 0x7f5f10160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 >> v2:172.21.15.172:3300/0 conn(0x7f5f101600b0 msgr2=0x7f5f100cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 --2- 172.21.15.89:0/4082411948 >> v2:172.21.15.172:3300/0 conn(0x7f5f101600b0 0x7f5f100cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100ca610 con 0x7f5f10160bb0 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.630+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f5f0c001670 con 0x7f5f10160bb0 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 >> v1:172.21.15.89:6789/0 conn(0x7f5f10160bb0 legacy=0x7f5f100b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a00 0x7f5efc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc004430 0x7f5efc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 0x7f5efc004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10156e80 con 0x7f5efc004430 2023-07-06T02:23:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100ca370 con 0x7f5efc003a00 2023-07-06T02:23:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc00ba10 con 0x7f5f10156ab0 2023-07-06T02:23:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f14a89700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc004430 0x7f5efc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f1528a700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a00 0x7f5efc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f15a8b700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 0x7f5efc004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f14a89700 1 -- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 msgr2=0x7f5efc004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f14a89700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10156ab0 0x7f5efc004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f14a89700 1 -- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a00 msgr2=0x7f5efc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f14a89700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5efc003a00 0x7f5efc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f14a89700 1 -- 172.21.15.89:0/4082411948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0035b0 con 0x7f5efc004430 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f14a89700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc004430 0x7f5efc0048f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c320 tx=0x7f5efc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f298214e7908972b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00ce60 con 0x7f5efc004430 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c005960 con 0x7f5efc004430 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.631+0000 7f5f07fff700 1 -- 172.21.15.89:0/4082411948 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c005d10 con 0x7f5efc004430 2023-07-06T02:23:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.632+0000 7f5f174ee700 1 -- 172.21.15.89:0/4082411948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc004430 msgr2=0x7f5efc0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.632+0000 7f5f174ee700 1 --2- 172.21.15.89:0/4082411948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5efc004430 0x7f5efc0048f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c320 tx=0x7f5efc005880 comp rx=0 tx=0).stop 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.632+0000 7f5f15a8b700 1 -- 172.21.15.89:0/4082411948 reap_dead start 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.632+0000 7f5f174ee700 1 -- 172.21.15.89:0/4082411948 shutdown_connections 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.632+0000 7f5f174ee700 1 -- 172.21.15.89:0/4082411948 >> 172.21.15.89:0/4082411948 conn(0x7f5f100cc490 msgr2=0x7f5f100cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.632+0000 7f5f174ee700 1 -- 172.21.15.89:0/4082411948 shutdown_connections 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.632+0000 7f5f174ee700 1 -- 172.21.15.89:0/4082411948 wait complete. 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.633+0000 7f5f174ee700 1 Processor -- start 2023-07-06T02:23:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.633+0000 7f5f174ee700 1 -- start start 2023-07-06T02:23:51.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f174ee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004430 0x7f5f1014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f174ee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c5a70 0x7f5f100ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f174ee700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100c4070 0x7f5f1015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f174ee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100c64f0 con 0x7f5f100c5a70 2023-07-06T02:23:51.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f174ee700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100b3d80 con 0x7f5f100c4070 2023-07-06T02:23:51.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f174ee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100c6210 con 0x7f5efc004430 2023-07-06T02:23:51.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c5a70 0x7f5f100ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c5a70 0x7f5f100ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34100/0 (socket says 172.21.15.89:34100) 2023-07-06T02:23:51.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 -- 172.21.15.89:0/2173588412 learned_addr learned my addr 172.21.15.89:0/2173588412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:51.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f15a8b700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100c4070 0x7f5f1015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f1528a700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004430 0x7f5f1014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 -- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004430 msgr2=0x7f5f1014ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004430 0x7f5f1014ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 -- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100c4070 msgr2=0x7f5f1015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100c4070 0x7f5f1015ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 -- 172.21.15.89:0/2173588412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100b36c0 con 0x7f5f100c5a70 2023-07-06T02:23:51.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f14a89700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c5a70 0x7f5f100ab8f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c910 tx=0x7f5f0c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f05ffb700 1 -- 172.21.15.89:0/2173588412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c001680 con 0x7f5f100c5a70 2023-07-06T02:23:51.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.634+0000 7f5f05ffb700 1 -- 172.21.15.89:0/2173588412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c004a80 con 0x7f5f100c5a70 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.635+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f100b39a0 con 0x7f5f100c5a70 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.635+0000 7f5f05ffb700 1 -- 172.21.15.89:0/2173588412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c004e30 con 0x7f5f100c5a70 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.635+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f100c5ee0 con 0x7f5f100c5a70 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.636+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f100b3520 con 0x7f5f100c5a70 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.636+0000 7f5f05ffb700 1 -- 172.21.15.89:0/2173588412 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f0c0070d0 con 0x7f5f100c5a70 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.636+0000 7f5f05ffb700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ace0 0x7f5ef406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.637+0000 7f5f05ffb700 1 -- 172.21.15.89:0/2173588412 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5f0c08ac20 con 0x7f5f100c5a70 2023-07-06T02:23:51.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.637+0000 7f5f1528a700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ace0 0x7f5ef406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.637+0000 7f5f1528a700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ace0 0x7f5ef406d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009a60 tx=0x7f5f00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.641+0000 7f5f05ffb700 1 -- 172.21.15.89:0/2173588412 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f100b3520 con 0x7f5f100c5a70 2023-07-06T02:23:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.827+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f100b3520 con 0x7f5f100c5a70 2023-07-06T02:23:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.827+0000 7f5f05ffb700 1 -- 172.21.15.89:0/2173588412 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f100b3520 con 0x7f5f100c5a70 2023-07-06T02:23:51.829 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:51.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ace0 msgr2=0x7f5ef406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ace0 0x7f5ef406d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009a60 tx=0x7f5f00008040 comp rx=0 tx=0).stop 2023-07-06T02:23:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c5a70 msgr2=0x7f5f100ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c5a70 0x7f5f100ab8f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c910 tx=0x7f5f0c001a00 comp rx=0 tx=0).stop 2023-07-06T02:23:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 shutdown_connections 2023-07-06T02:23:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100c4070 0x7f5f1015ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc004430 0x7f5f1014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5ef406ace0 0x7f5ef406d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 --2- 172.21.15.89:0/2173588412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f100c5a70 0x7f5f100ab8f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 >> 172.21.15.89:0/2173588412 conn(0x7f5f100cc490 msgr2=0x7f5f100ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 shutdown_connections 2023-07-06T02:23:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.829+0000 7f5f174ee700 1 -- 172.21.15.89:0/2173588412 wait complete. 2023-07-06T02:23:51.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:51.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.964+0000 7fc28fbcc700 1 Processor -- start 2023-07-06T02:23:51.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.964+0000 7fc28fbcc700 1 -- start start 2023-07-06T02:23:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.965+0000 7fc28fbcc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 0x7fc2880cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.965+0000 7fc28fbcc700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880cab50 con 0x7fc2880cbda0 2023-07-06T02:23:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.965+0000 7fc28fbcc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28815db70 con 0x7fc2880b1ba0 2023-07-06T02:23:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.965+0000 7fc28fbcc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28815de50 con 0x7fc28815c690 2023-07-06T02:23:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.965+0000 7fc28d167700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc2880cbda0 0x7fc2880b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59008/0 (socket says 172.21.15.89:59008) 2023-07-06T02:23:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.965+0000 7fc28d167700 1 -- 172.21.15.89:0/801825089 learned_addr learned my addr 172.21.15.89:0/801825089 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.965+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557715639 0 0) 0x7fc2880cab50 con 0x7fc2880cbda0 2023-07-06T02:23:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d968700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 0x7fc2880cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2740035b0 con 0x7fc2880cbda0 2023-07-06T02:23:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1396507049 0 0) 0x7fc2740035b0 con 0x7fc2880cbda0 2023-07-06T02:23:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 msgr2=0x7fc2880cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 0x7fc2880cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 >> v1:172.21.15.172:6789/0 conn(0x7fc2880b1ba0 legacy=0x7fc28806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880cad80 con 0x7fc2880cbda0 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fc284001670 con 0x7fc2880cbda0 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 >> v1:172.21.15.89:6789/0 conn(0x7fc2880cbda0 legacy=0x7fc2880b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 0x7fc274003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc274003fb0 0x7fc274004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274004d90 0x7fc274005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28815de50 con 0x7fc274003fb0 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400baf0 con 0x7fc274004d90 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400bcb0 con 0x7fc28815c690 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d167700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc274003fb0 0x7fc274004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d968700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 0x7fc274003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28e169700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274004d90 0x7fc274005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d167700 1 -- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 msgr2=0x7fc274003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d167700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28815c690 0x7fc274003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d167700 1 -- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274004d90 msgr2=0x7fc274005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d167700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274004d90 0x7fc274005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d167700 1 -- 172.21.15.89:0/801825089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740035b0 con 0x7fc274003fb0 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.966+0000 7fc28d167700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc274003fb0 0x7fc274004850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c210 tx=0x7fc274005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c58d76ec819211 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284007c10 con 0x7fc274003fb0 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28400cdb0 con 0x7fc274003fb0 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28c966700 1 -- 172.21.15.89:0/801825089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284005ba0 con 0x7fc274003fb0 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28fbcc700 1 -- 172.21.15.89:0/801825089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc274003fb0 msgr2=0x7fc274004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28fbcc700 1 --2- 172.21.15.89:0/801825089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc274003fb0 0x7fc274004850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c210 tx=0x7fc274005880 comp rx=0 tx=0).stop 2023-07-06T02:23:51.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28e169700 1 -- 172.21.15.89:0/801825089 reap_dead start 2023-07-06T02:23:51.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28fbcc700 1 -- 172.21.15.89:0/801825089 shutdown_connections 2023-07-06T02:23:51.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28fbcc700 1 -- 172.21.15.89:0/801825089 >> 172.21.15.89:0/801825089 conn(0x7fc2880cc490 msgr2=0x7fc2880cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:51.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28fbcc700 1 -- 172.21.15.89:0/801825089 shutdown_connections 2023-07-06T02:23:51.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.967+0000 7fc28fbcc700 1 -- 172.21.15.89:0/801825089 wait complete. 2023-07-06T02:23:51.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.968+0000 7fc28fbcc700 1 Processor -- start 2023-07-06T02:23:51.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.968+0000 7fc28fbcc700 1 -- start start 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28fbcc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274003fb0 0x7fc2880c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28fbcc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2881554f0 0x7fc2880c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28fbcc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2881583f0 0x7fc288158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28fbcc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c7980 con 0x7fc2881554f0 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28fbcc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288158da0 con 0x7fc274003fb0 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28fbcc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c3e20 con 0x7fc2881583f0 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2881554f0 0x7fc2880c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2881554f0 0x7fc2880c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34120/0 (socket says 172.21.15.89:34120) 2023-07-06T02:23:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 -- 172.21.15.89:0/824260842 learned_addr learned my addr 172.21.15.89:0/824260842 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d968700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274003fb0 0x7fc2880c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28e169700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2881583f0 0x7fc288158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 -- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2881583f0 msgr2=0x7fc288158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2881583f0 0x7fc288158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 -- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274003fb0 msgr2=0x7fc2880c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:51.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274003fb0 0x7fc2880c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.969+0000 7fc28d167700 1 -- 172.21.15.89:0/824260842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880c72c0 con 0x7fc2881554f0 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.970+0000 7fc28d167700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2881554f0 0x7fc2880c3540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c800 tx=0x7fc284001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.970+0000 7fc27e7fc700 1 -- 172.21.15.89:0/824260842 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284001640 con 0x7fc2881554f0 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.970+0000 7fc27e7fc700 1 -- 172.21.15.89:0/824260842 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2840070d0 con 0x7fc2881554f0 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.970+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2880c75a0 con 0x7fc2881554f0 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.970+0000 7fc27e7fc700 1 -- 172.21.15.89:0/824260842 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2840074a0 con 0x7fc2881554f0 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.970+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880c6250 con 0x7fc2881554f0 2023-07-06T02:23:51.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.971+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288074270 con 0x7fc2881554f0 2023-07-06T02:23:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.971+0000 7fc27e7fc700 1 -- 172.21.15.89:0/824260842 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc284019030 con 0x7fc2881554f0 2023-07-06T02:23:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.971+0000 7fc27e7fc700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc26c06ace0 0x7fc26c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.972+0000 7fc27e7fc700 1 -- 172.21.15.89:0/824260842 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc28408a590 con 0x7fc2881554f0 2023-07-06T02:23:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.972+0000 7fc28d968700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc26c06ace0 0x7fc26c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.972+0000 7fc28d968700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc26c06ace0 0x7fc26c06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b04d0 tx=0x7fc278008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:51.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:51.976+0000 7fc27e7fc700 1 -- 172.21.15.89:0/824260842 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc284055320 con 0x7fc2881554f0 2023-07-06T02:23:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.163+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2880aec30 con 0x7fc2881554f0 2023-07-06T02:23:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.163+0000 7fc27e7fc700 1 -- 172.21.15.89:0/824260842 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc284058940 con 0x7fc2881554f0 2023-07-06T02:23:52.165 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc26c06ace0 msgr2=0x7fc26c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc26c06ace0 0x7fc26c06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b04d0 tx=0x7fc278008040 comp rx=0 tx=0).stop 2023-07-06T02:23:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2881554f0 msgr2=0x7fc2880c3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2881554f0 0x7fc2880c3540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c800 tx=0x7fc284001a00 comp rx=0 tx=0).stop 2023-07-06T02:23:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 shutdown_connections 2023-07-06T02:23:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2881583f0 0x7fc288158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc274003fb0 0x7fc2880c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fc26c06ace0 0x7fc26c06d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 --2- 172.21.15.89:0/824260842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2881554f0 0x7fc2880c3540 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 >> 172.21.15.89:0/824260842 conn(0x7fc2880cc490 msgr2=0x7fc28806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 shutdown_connections 2023-07-06T02:23:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.165+0000 7fc28fbcc700 1 -- 172.21.15.89:0/824260842 wait complete. 2023-07-06T02:23:52.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.300+0000 7fac68585700 1 Processor -- start 2023-07-06T02:23:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.300+0000 7fac68585700 1 -- start start 2023-07-06T02:23:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac68585700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fac60156ab0 0x7fac60160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac68585700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac601600b0 0x7fac600cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac68585700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac600cb3d0 con 0x7fac60160bb0 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac68585700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac60156e80 con 0x7fac601600b0 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac68585700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac600ca370 con 0x7fac60156ab0 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66b22700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac601600b0 0x7fac600cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac65b20700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fac60160bb0 0x7fac600b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59028/0 (socket says 172.21.15.89:59028) 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66b22700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac601600b0 0x7fac600cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34130/0 (socket says 172.21.15.89:34130) 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac65b20700 1 -- 172.21.15.89:0/3606188842 learned_addr learned my addr 172.21.15.89:0/3606188842 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66321700 1 --2- 172.21.15.89:0/3606188842 >> v2:172.21.15.172:3300/0 conn(0x7fac60156ab0 0x7fac60160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66b22700 1 -- 172.21.15.89:0/3606188842 >> v2:172.21.15.172:3300/0 conn(0x7fac60156ab0 msgr2=0x7fac60160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66b22700 1 --2- 172.21.15.89:0/3606188842 >> v2:172.21.15.172:3300/0 conn(0x7fac60156ab0 0x7fac60160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66b22700 1 -- 172.21.15.89:0/3606188842 >> v1:172.21.15.89:6789/0 conn(0x7fac60160bb0 legacy=0x7fac600b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66b22700 1 -- 172.21.15.89:0/3606188842 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600ca610 con 0x7fac601600b0 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac6531f700 1 -- 172.21.15.89:0/3606188842 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778750636 0 0) 0x7fac600cb3d0 con 0x7fac60160bb0 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac66b22700 1 --2- 172.21.15.89:0/3606188842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600cd700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fac6006e9e0 tx=0x7fac5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136f28de000bbfc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac6531f700 1 -- 172.21.15.89:0/3606188842 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800e070 con 0x7fac601600b0 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac6531f700 1 -- 172.21.15.89:0/3606188842 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5800bb20 con 0x7fac601600b0 2023-07-06T02:23:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.301+0000 7fac6531f700 1 -- 172.21.15.89:0/3606188842 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58012430 con 0x7fac601600b0 2023-07-06T02:23:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 -- 172.21.15.89:0/3606188842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 msgr2=0x7fac600cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 --2- 172.21.15.89:0/3606188842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600cd700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fac6006e9e0 tx=0x7fac5800b330 comp rx=0 tx=0).stop 2023-07-06T02:23:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 -- 172.21.15.89:0/3606188842 shutdown_connections 2023-07-06T02:23:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 --2- 172.21.15.89:0/3606188842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600cd700 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 --2- 172.21.15.89:0/3606188842 >> v2:172.21.15.172:3300/0 conn(0x7fac60156ab0 0x7fac60160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 -- 172.21.15.89:0/3606188842 >> 172.21.15.89:0/3606188842 conn(0x7fac600cc490 msgr2=0x7fac600cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 -- 172.21.15.89:0/3606188842 shutdown_connections 2023-07-06T02:23:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.302+0000 7fac68585700 1 -- 172.21.15.89:0/3606188842 wait complete. 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.303+0000 7fac68585700 1 Processor -- start 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.303+0000 7fac68585700 1 -- start start 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac68585700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fac60156ab0 0x7fac600b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac68585700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac68585700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac601f5820 0x7fac60155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac68585700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac600c9af0 con 0x7fac601600b0 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac68585700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac600ae740 con 0x7fac601f5820 2023-07-06T02:23:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac68585700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac600b4360 con 0x7fac60156ab0 2023-07-06T02:23:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34134/0 (socket says 172.21.15.89:34134) 2023-07-06T02:23:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 -- 172.21.15.89:0/2886933413 learned_addr learned my addr 172.21.15.89:0/2886933413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 -- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fac60156ab0 msgr2=0x7fac600b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac66b22700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac601f5820 0x7fac60155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac66321700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fac60156ab0 0x7fac600b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fac60156ab0 0x7fac600b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 -- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac601f5820 msgr2=0x7fac60155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac601f5820 0x7fac60155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.304+0000 7fac65b20700 1 -- 172.21.15.89:0/2886933413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58009a70 con 0x7fac601600b0 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.305+0000 7fac65b20700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600c95b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00c540 tx=0x7fac5c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.305+0000 7fac56ffd700 1 -- 172.21.15.89:0/2886933413 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c004c80 con 0x7fac601600b0 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.305+0000 7fac56ffd700 1 -- 172.21.15.89:0/2886933413 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c004de0 con 0x7fac601600b0 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.305+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac600b4640 con 0x7fac601600b0 2023-07-06T02:23:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.305+0000 7fac56ffd700 1 -- 172.21.15.89:0/2886933413 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c007310 con 0x7fac601600b0 2023-07-06T02:23:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.305+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac600aea20 con 0x7fac601600b0 2023-07-06T02:23:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.306+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac600b4b60 con 0x7fac601600b0 2023-07-06T02:23:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.306+0000 7fac56ffd700 1 -- 172.21.15.89:0/2886933413 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac5c007470 con 0x7fac601600b0 2023-07-06T02:23:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.306+0000 7fac56ffd700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fac4406ace0 0x7fac4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.307+0000 7fac56ffd700 1 -- 172.21.15.89:0/2886933413 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fac5c0895b0 con 0x7fac601600b0 2023-07-06T02:23:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.307+0000 7fac66321700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fac4406ace0 0x7fac4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.307+0000 7fac66321700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fac4406ace0 0x7fac4406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac50009a10 tx=0x7fac50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.311+0000 7fac56ffd700 1 -- 172.21.15.89:0/2886933413 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac600b4b60 con 0x7fac601600b0 2023-07-06T02:23:52.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.502+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac600b4b60 con 0x7fac601600b0 2023-07-06T02:23:52.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.502+0000 7fac56ffd700 1 -- 172.21.15.89:0/2886933413 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac600b4b60 con 0x7fac601600b0 2023-07-06T02:23:52.504 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:52.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fac4406ace0 msgr2=0x7fac4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fac4406ace0 0x7fac4406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac50009a10 tx=0x7fac50008040 comp rx=0 tx=0).stop 2023-07-06T02:23:52.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 msgr2=0x7fac600c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600c95b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00c540 tx=0x7fac5c00c900 comp rx=0 tx=0).stop 2023-07-06T02:23:52.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 shutdown_connections 2023-07-06T02:23:52.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac601f5820 0x7fac60155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fac60156ab0 0x7fac600b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fac4406ace0 0x7fac4406d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 --2- 172.21.15.89:0/2886933413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac601600b0 0x7fac600c95b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.504+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 >> 172.21.15.89:0/2886933413 conn(0x7fac600cc490 msgr2=0x7fac6015dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:52.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.505+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 shutdown_connections 2023-07-06T02:23:52.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.505+0000 7fac68585700 1 -- 172.21.15.89:0/2886933413 wait complete. 2023-07-06T02:23:52.518 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.639+0000 7f889b6bd700 1 Processor -- start 2023-07-06T02:23:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.639+0000 7f889b6bd700 1 -- start start 2023-07-06T02:23:52.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f889b6bd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f889415c690 0x7f88940cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f889b6bd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f88940cbda0 0x7f88940b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f889b6bd700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88940cab50 con 0x7f88940b19d0 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f889b6bd700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889415de50 con 0x7f88940cbda0 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f889b6bd700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88940b1dc0 con 0x7f889415c690 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f88940cbda0 0x7f88940b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f88940cbda0 0x7f88940b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34140/0 (socket says 172.21.15.89:34140) 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 -- 172.21.15.89:0/2025490859 learned_addr learned my addr 172.21.15.89:0/2025490859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:52.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8899459700 1 --2- 172.21.15.89:0/2025490859 >> v2:172.21.15.172:3300/0 conn(0x7f889415c690 0x7f88940cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 -- 172.21.15.89:0/2025490859 >> v2:172.21.15.172:3300/0 conn(0x7f889415c690 msgr2=0x7f88940cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:52.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 --2- 172.21.15.89:0/2025490859 >> v2:172.21.15.172:3300/0 conn(0x7f889415c690 0x7f88940cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 -- 172.21.15.89:0/2025490859 >> v1:172.21.15.172:6789/0 conn(0x7f88940b19d0 legacy=0x7f889406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:52.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 -- 172.21.15.89:0/2025490859 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889415db70 con 0x7f88940cbda0 2023-07-06T02:23:52.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.640+0000 7f8898c58700 1 --2- 172.21.15.89:0/2025490859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88940cbda0 0x7f88940b1490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f88940b2600 tx=0x7f8884009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a306806df39bdc90 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f888bfff700 1 -- 172.21.15.89:0/2025490859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884004230 con 0x7f88940cbda0 2023-07-06T02:23:52.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f888bfff700 1 -- 172.21.15.89:0/2025490859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8884004390 con 0x7f88940cbda0 2023-07-06T02:23:52.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f888bfff700 1 -- 172.21.15.89:0/2025490859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840036a0 con 0x7f88940cbda0 2023-07-06T02:23:52.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 -- 172.21.15.89:0/2025490859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88940cbda0 msgr2=0x7f88940b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 --2- 172.21.15.89:0/2025490859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88940cbda0 0x7f88940b1490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f88940b2600 tx=0x7f8884009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 -- 172.21.15.89:0/2025490859 shutdown_connections 2023-07-06T02:23:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 --2- 172.21.15.89:0/2025490859 >> v2:172.21.15.172:3300/0 conn(0x7f889415c690 0x7f88940cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 --2- 172.21.15.89:0/2025490859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88940cbda0 0x7f88940b1490 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 -- 172.21.15.89:0/2025490859 >> 172.21.15.89:0/2025490859 conn(0x7f88940cc490 msgr2=0x7f88940cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 -- 172.21.15.89:0/2025490859 shutdown_connections 2023-07-06T02:23:52.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.641+0000 7f889b6bd700 1 -- 172.21.15.89:0/2025490859 wait complete. 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.643+0000 7f889b6bd700 1 Processor -- start 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.643+0000 7f889b6bd700 1 -- start start 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f889b6bd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88940b19d0 0x7f88940b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f889b6bd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88940cbda0 0x7f88940c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f889b6bd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f889415c690 0x7f88940c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f889b6bd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88940c6430 con 0x7f889415c690 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f889b6bd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88941f02d0 con 0x7f88940cbda0 2023-07-06T02:23:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f889b6bd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88940c4590 con 0x7f88940b19d0 2023-07-06T02:23:52.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f889415c690 0x7f88940c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f889415c690 0x7f88940c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34150/0 (socket says 172.21.15.89:34150) 2023-07-06T02:23:52.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 -- 172.21.15.89:0/3878636958 learned_addr learned my addr 172.21.15.89:0/3878636958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:52.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8898c58700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88940cbda0 0x7f88940c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899459700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88940b19d0 0x7f88940b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 -- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88940b19d0 msgr2=0x7f88940b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88940b19d0 0x7f88940b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 -- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88940cbda0 msgr2=0x7f88940c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88940cbda0 0x7f88940c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 -- 172.21.15.89:0/3878636958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8884009550 con 0x7f889415c690 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8899c5a700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f889415c690 0x7f88940c5ef0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b970 tx=0x7f888c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8889ffb700 1 -- 172.21.15.89:0/3878636958 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00c930 con 0x7f889415c690 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8889ffb700 1 -- 172.21.15.89:0/3878636958 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c00cad0 con 0x7f889415c690 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f8889ffb700 1 -- 172.21.15.89:0/3878636958 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00ce80 con 0x7f889415c690 2023-07-06T02:23:52.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.644+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88940c53f0 con 0x7f889415c690 2023-07-06T02:23:52.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.645+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88940c4ac0 con 0x7f889415c690 2023-07-06T02:23:52.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.646+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894074270 con 0x7f889415c690 2023-07-06T02:23:52.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.646+0000 7f8889ffb700 1 -- 172.21.15.89:0/3878636958 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f888c014440 con 0x7f889415c690 2023-07-06T02:23:52.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.646+0000 7f8889ffb700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f887806ace0 0x7f887806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.646+0000 7f8889ffb700 1 -- 172.21.15.89:0/3878636958 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f888c0893b0 con 0x7f889415c690 2023-07-06T02:23:52.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.646+0000 7f8899459700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f887806ace0 0x7f887806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.647+0000 7f8899459700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f887806ace0 0x7f887806d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8890009930 tx=0x7f8890008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.651+0000 7f8889ffb700 1 -- 172.21.15.89:0/3878636958 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f888c054140 con 0x7f889415c690 2023-07-06T02:23:52.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.833+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f88940c68a0 con 0x7f889415c690 2023-07-06T02:23:52.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.833+0000 7f8889ffb700 1 -- 172.21.15.89:0/3878636958 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f888c057760 con 0x7f889415c690 2023-07-06T02:23:52.837 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:52.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f887806ace0 msgr2=0x7f887806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f887806ace0 0x7f887806d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8890009930 tx=0x7f8890008040 comp rx=0 tx=0).stop 2023-07-06T02:23:52.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f889415c690 msgr2=0x7f88940c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f889415c690 0x7f88940c5ef0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b970 tx=0x7f888c00bc80 comp rx=0 tx=0).stop 2023-07-06T02:23:52.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 shutdown_connections 2023-07-06T02:23:52.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f887806ace0 0x7f887806d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f889415c690 0x7f88940c5ef0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88940b19d0 0x7f88940b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 --2- 172.21.15.89:0/3878636958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88940cbda0 0x7f88940c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 >> 172.21.15.89:0/3878636958 conn(0x7f88940cc490 msgr2=0x7f88940cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:52.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 shutdown_connections 2023-07-06T02:23:52.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.835+0000 7f889b6bd700 1 -- 172.21.15.89:0/3878636958 wait complete. 2023-07-06T02:23:52.850 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T02:23:52.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.968+0000 7f9367e49700 1 Processor -- start 2023-07-06T02:23:52.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.968+0000 7f9367e49700 1 -- start start 2023-07-06T02:23:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.969+0000 7f9367e49700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93600b1490 0x7f93600b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.969+0000 7f9367e49700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93600cab50 con 0x7f936015c690 2023-07-06T02:23:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.969+0000 7f9367e49700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936015db70 con 0x7f936006bfa0 2023-07-06T02:23:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.969+0000 7f9367e49700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936015de50 con 0x7f93600b1490 2023-07-06T02:23:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.969+0000 7f9365be5700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f936015c690 0x7f93600cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59056/0 (socket says 172.21.15.89:59056) 2023-07-06T02:23:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.969+0000 7f9365be5700 1 -- 172.21.15.89:0/1501189441 learned_addr learned my addr 172.21.15.89:0/1501189441 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93653e4700 1 --2- 172.21.15.89:0/1501189441 >> v2:172.21.15.172:3300/0 conn(0x7f93600b1490 0x7f93600b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462454727 0 0) 0x7f93600cab50 con 0x7f936015c690 2023-07-06T02:23:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f934c0035b0 con 0x7f936015c690 2023-07-06T02:23:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2273327466 0 0) 0x7f934c0035b0 con 0x7f936015c690 2023-07-06T02:23:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 >> v2:172.21.15.172:3300/0 conn(0x7f93600b1490 msgr2=0x7f93600b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 --2- 172.21.15.89:0/1501189441 >> v2:172.21.15.172:3300/0 conn(0x7f93600b1490 0x7f93600b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 >> v1:172.21.15.172:6789/0 conn(0x7f936006bfa0 legacy=0x7f936006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600cad80 con 0x7f936015c690 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f9350001670 con 0x7f936015c690 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 >> v1:172.21.15.89:6789/0 conn(0x7f936015c690 legacy=0x7f93600cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c003a00 0x7f934c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c004850 0x7f934c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f934c005200 0x7f934c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936015de50 con 0x7f934c005200 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c00e290 con 0x7f934c003a00 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c00e450 con 0x7f934c004850 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93663e6700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f934c005200 0x7f934c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f9365be5700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c003a00 0x7f934c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93653e4700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c004850 0x7f934c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93663e6700 1 -- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c004850 msgr2=0x7f934c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93663e6700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c004850 0x7f934c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93663e6700 1 -- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c003a00 msgr2=0x7f934c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93663e6700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c003a00 0x7f934c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93663e6700 1 -- 172.21.15.89:0/1501189441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0035b0 con 0x7f934c005200 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.970+0000 7f93663e6700 1 -- 172.21.15.89:0/1501189441 reap_dead start 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f93663e6700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f934c005200 0x7f934c00ba10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f935800c190 tx=0x7f934c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606a7719d9ba2503 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358001db0 con 0x7f934c005200 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935800ccf0 con 0x7f934c005200 2023-07-06T02:23:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9364be3700 1 -- 172.21.15.89:0/1501189441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358003c40 con 0x7f934c005200 2023-07-06T02:23:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9367e49700 1 -- 172.21.15.89:0/1501189441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f934c005200 msgr2=0x7f934c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9367e49700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f934c005200 0x7f934c00ba10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f935800c190 tx=0x7f934c005880 comp rx=0 tx=0).stop 2023-07-06T02:23:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9367e49700 1 -- 172.21.15.89:0/1501189441 shutdown_connections 2023-07-06T02:23:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9367e49700 1 --2- 172.21.15.89:0/1501189441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f934c005200 0x7f934c00ba10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9367e49700 1 -- 172.21.15.89:0/1501189441 >> 172.21.15.89:0/1501189441 conn(0x7f93600cc490 msgr2=0x7f93600cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9367e49700 1 -- 172.21.15.89:0/1501189441 shutdown_connections 2023-07-06T02:23:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.971+0000 7f9367e49700 1 -- 172.21.15.89:0/1501189441 wait complete. 2023-07-06T02:23:52.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.972+0000 7f9367e49700 1 Processor -- start 2023-07-06T02:23:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f9367e49700 1 -- start start 2023-07-06T02:23:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f9367e49700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c005200 0x7f936015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f9367e49700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9360156040 0x7f936014ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f9367e49700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93600b2860 0x7f936015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f9367e49700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b2cd0 con 0x7f9360156040 2023-07-06T02:23:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f9367e49700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600ac9a0 con 0x7f93600b2860 2023-07-06T02:23:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f9367e49700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600acc80 con 0x7f934c005200 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f93653e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9360156040 0x7f936014ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f93653e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9360156040 0x7f936014ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34168/0 (socket says 172.21.15.89:34168) 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.973+0000 7f93653e4700 1 -- 172.21.15.89:0/4194786269 learned_addr learned my addr 172.21.15.89:0/4194786269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93663e6700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93600b2860 0x7f936015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f9365be5700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c005200 0x7f936015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93653e4700 1 -- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c005200 msgr2=0x7f936015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93653e4700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c005200 0x7f936015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93653e4700 1 -- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93600b2860 msgr2=0x7f936015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93653e4700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93600b2860 0x7f936015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93653e4700 1 -- 172.21.15.89:0/4194786269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600acf60 con 0x7f9360156040 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f9365be5700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c005200 0x7f936015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93653e4700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9360156040 0x7f936014ef10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f935c0119a0 tx=0x7f935c011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93567fc700 1 -- 172.21.15.89:0/4194786269 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00f940 con 0x7f9360156040 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93567fc700 1 -- 172.21.15.89:0/4194786269 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c00faa0 con 0x7f9360156040 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93600ad240 con 0x7f9360156040 2023-07-06T02:23:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f93567fc700 1 -- 172.21.15.89:0/4194786269 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00fe30 con 0x7f9360156040 2023-07-06T02:23:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.974+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93600c9400 con 0x7f9360156040 2023-07-06T02:23:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.975+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9360074270 con 0x7f9360156040 2023-07-06T02:23:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.975+0000 7f93567fc700 1 -- 172.21.15.89:0/4194786269 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f935c014440 con 0x7f9360156040 2023-07-06T02:23:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.976+0000 7f93567fc700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f934406ace0 0x7f934406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.976+0000 7f9365be5700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f934406ace0 0x7f934406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.976+0000 7f93567fc700 1 -- 172.21.15.89:0/4194786269 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f935c089e30 con 0x7f9360156040 2023-07-06T02:23:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.976+0000 7f9365be5700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f934406ace0 0x7f934406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9350007d10 tx=0x7f9350009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:52.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:52.980+0000 7f93567fc700 1 -- 172.21.15.89:0/4194786269 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f935c054bb0 con 0x7f9360156040 2023-07-06T02:23:53.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.182+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f936015b940 con 0x7f9360156040 2023-07-06T02:23:53.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.183+0000 7f93567fc700 1 -- 172.21.15.89:0/4194786269 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f935c0581d0 con 0x7f9360156040 2023-07-06T02:23:53.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f934406ace0 msgr2=0x7f934406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f934406ace0 0x7f934406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9350007d10 tx=0x7f9350009040 comp rx=0 tx=0).stop 2023-07-06T02:23:53.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9360156040 msgr2=0x7f936014ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9360156040 0x7f936014ef10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f935c0119a0 tx=0x7f935c011cb0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 shutdown_connections 2023-07-06T02:23:53.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93600b2860 0x7f936015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f934c005200 0x7f936015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f934406ace0 0x7f934406d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 --2- 172.21.15.89:0/4194786269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9360156040 0x7f936014ef10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 >> 172.21.15.89:0/4194786269 conn(0x7f93600cc490 msgr2=0x7f936006c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:53.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 shutdown_connections 2023-07-06T02:23:53.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.185+0000 7f9367e49700 1 -- 172.21.15.89:0/4194786269 wait complete. 2023-07-06T02:23:53.190 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-07-06T02:23:53.199 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-07-06T02:23:53.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:53.200 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-07-06T02:23:53.251 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:23:53.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:23:53.252 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T02:23:53.271 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T02:23:53.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.390+0000 7f707da95700 1 Processor -- start 2023-07-06T02:23:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.390+0000 7f707da95700 1 -- start start 2023-07-06T02:23:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f707da95700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 0x7f70780b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f707da95700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70780cab50 con 0x7f707815c690 2023-07-06T02:23:53.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f707da95700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707815db70 con 0x7f70780b1490 2023-07-06T02:23:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f707da95700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707815de50 con 0x7f707806c6b0 2023-07-06T02:23:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f70777fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f707815c690 0x7f70780cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59076/0 (socket says 172.21.15.89:59076) 2023-07-06T02:23:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f70777fe700 1 -- 172.21.15.89:0/1752467345 learned_addr learned my addr 172.21.15.89:0/1752467345 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7077fff700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 0x7f70780b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008622787 0 0) 0x7f70780cab50 con 0x7f707815c690 2023-07-06T02:23:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70640057c0 con 0x7f707815c690 2023-07-06T02:23:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92639891 0 0) 0x7f70640057c0 con 0x7f707815c690 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 msgr2=0x7f70780b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 0x7f70780b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 >> v1:172.21.15.172:6789/0 conn(0x7f70780b1490 legacy=0x7f707806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780cad80 con 0x7f707815c690 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.391+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f7068001670 con 0x7f707815c690 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 >> v1:172.21.15.89:6789/0 conn(0x7f707815c690 legacy=0x7f70780cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7064005c40 0x7f7064006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 0x7f7064006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064006fa0 0x7f7064007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707815de50 con 0x7f7064005c40 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706400dc20 con 0x7f7064006fa0 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706400ddb0 con 0x7f707806c6b0 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f70777fe700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7064005c40 0x7f7064006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f706effd700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 0x7f7064006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7077fff700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064006fa0 0x7f7064007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f70777fe700 1 -- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 msgr2=0x7f7064006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f70777fe700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f707806c6b0 0x7f7064006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f70777fe700 1 -- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064006fa0 msgr2=0x7f7064007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f70777fe700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064006fa0 0x7f7064007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f70777fe700 1 -- 172.21.15.89:0/1752467345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640057c0 con 0x7f7064005c40 2023-07-06T02:23:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f70777fe700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7064005c40 0x7f7064006010 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f706800c2f0 tx=0x7f7064007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbb5e584d725a59 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706800ce30 con 0x7f7064005c40 2023-07-06T02:23:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068005960 con 0x7f7064005c40 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.392+0000 7f7076ffd700 1 -- 172.21.15.89:0/1752467345 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068005cf0 con 0x7f7064005c40 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.393+0000 7f707da95700 1 -- 172.21.15.89:0/1752467345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7064005c40 msgr2=0x7f7064006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.393+0000 7f707da95700 1 --2- 172.21.15.89:0/1752467345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7064005c40 0x7f7064006010 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f706800c2f0 tx=0x7f7064007a90 comp rx=0 tx=0).stop 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.393+0000 7f7077fff700 1 -- 172.21.15.89:0/1752467345 reap_dead start 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.393+0000 7f707da95700 1 -- 172.21.15.89:0/1752467345 shutdown_connections 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.393+0000 7f707da95700 1 -- 172.21.15.89:0/1752467345 >> 172.21.15.89:0/1752467345 conn(0x7f70780cc490 msgr2=0x7f70780cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.393+0000 7f707da95700 1 -- 172.21.15.89:0/1752467345 shutdown_connections 2023-07-06T02:23:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.393+0000 7f707da95700 1 -- 172.21.15.89:0/1752467345 wait complete. 2023-07-06T02:23:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.394+0000 7f707da95700 1 Processor -- start 2023-07-06T02:23:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.394+0000 7f707da95700 1 -- start start 2023-07-06T02:23:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f707da95700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064005c40 0x7f70780c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f707da95700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70780b4cb0 0x7f7078148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f707da95700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70780c8f20 0x7f70780c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f707da95700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780c98d0 con 0x7f70780b4cb0 2023-07-06T02:23:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f707da95700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780c40e0 con 0x7f7064005c40 2023-07-06T02:23:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f707da95700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780c8750 con 0x7f70780c8f20 2023-07-06T02:23:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70780b4cb0 0x7f7078148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70780b4cb0 0x7f7078148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34188/0 (socket says 172.21.15.89:34188) 2023-07-06T02:23:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 -- 172.21.15.89:0/2837509235 learned_addr learned my addr 172.21.15.89:0/2837509235 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f70777fe700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064005c40 0x7f70780c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f7077fff700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70780c8f20 0x7f70780c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 -- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70780c8f20 msgr2=0x7f70780c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70780c8f20 0x7f70780c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 -- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064005c40 msgr2=0x7f70780c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064005c40 0x7f70780c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.395+0000 7f706effd700 1 -- 172.21.15.89:0/2837509235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780c7bd0 con 0x7f70780b4cb0 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.396+0000 7f706effd700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70780b4cb0 0x7f7078148db0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f706000f810 tx=0x7f706000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.396+0000 7f7074ff9700 1 -- 172.21.15.89:0/2837509235 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70600052e0 con 0x7f70780b4cb0 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.396+0000 7f7074ff9700 1 -- 172.21.15.89:0/2837509235 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060005440 con 0x7f70780b4cb0 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.396+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780c7eb0 con 0x7f70780b4cb0 2023-07-06T02:23:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.396+0000 7f7074ff9700 1 -- 172.21.15.89:0/2837509235 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70600038c0 con 0x7f70780b4cb0 2023-07-06T02:23:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.396+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70780c8470 con 0x7f70780b4cb0 2023-07-06T02:23:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.397+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70780c4270 con 0x7f70780b4cb0 2023-07-06T02:23:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.397+0000 7f7074ff9700 1 -- 172.21.15.89:0/2837509235 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7060003a20 con 0x7f70780b4cb0 2023-07-06T02:23:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.398+0000 7f7074ff9700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f705806ad10 0x7f705806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.398+0000 7f7074ff9700 1 -- 172.21.15.89:0/2837509235 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7060089f50 con 0x7f70780b4cb0 2023-07-06T02:23:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.398+0000 7f70777fe700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f705806ad10 0x7f705806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.398+0000 7f70777fe700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f705806ad10 0x7f705806d1c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7068008fd0 tx=0x7f7068007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.402+0000 7f7074ff9700 1 -- 172.21.15.89:0/2837509235 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7060054d20 con 0x7f70780b4cb0 2023-07-06T02:23:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.638+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70780c4270 con 0x7f70780b4cb0 2023-07-06T02:23:53.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.639+0000 7f7074ff9700 1 -- 172.21.15.89:0/2837509235 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f70780c4270 con 0x7f70780b4cb0 2023-07-06T02:23:53.642 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:53.642 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f705806ad10 msgr2=0x7f705806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f705806ad10 0x7f705806d1c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7068008fd0 tx=0x7f7068007000 comp rx=0 tx=0).stop 2023-07-06T02:23:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70780b4cb0 msgr2=0x7f7078148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70780b4cb0 0x7f7078148db0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f706000f810 tx=0x7f706000fbd0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 shutdown_connections 2023-07-06T02:23:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70780c8f20 0x7f70780c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7064005c40 0x7f70780c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f705806ad10 0x7f705806d1c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 --2- 172.21.15.89:0/2837509235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70780b4cb0 0x7f7078148db0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 >> 172.21.15.89:0/2837509235 conn(0x7f70780cc490 msgr2=0x7f70780cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 shutdown_connections 2023-07-06T02:23:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.641+0000 7f707da95700 1 -- 172.21.15.89:0/2837509235 wait complete. 2023-07-06T02:23:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:53.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:53.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.772+0000 7fba39552700 1 Processor -- start 2023-07-06T02:23:53.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.772+0000 7fba39552700 1 -- start start 2023-07-06T02:23:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba39552700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba3406bdb0 0x7fba3406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba39552700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba340ceda0 0x7fba3415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba39552700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba340cd240 con 0x7fba3406b610 2023-07-06T02:23:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba39552700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba340cf280 con 0x7fba340ceda0 2023-07-06T02:23:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba39552700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba340cd490 con 0x7fba3406bdb0 2023-07-06T02:23:53.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba337fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba340ceda0 0x7fba3415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba32ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba3406bdb0 0x7fba3406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba337fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba340ceda0 0x7fba3415c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:32954/0 (socket says 172.21.15.89:32954) 2023-07-06T02:23:53.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba337fe700 1 -- 172.21.15.89:0/3577277608 learned_addr learned my addr 172.21.15.89:0/3577277608 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:53.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.773+0000 7fba327fc700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fba3406b610 0x7fba3406c180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:34854/0 (socket says 172.21.15.89:34854) 2023-07-06T02:23:53.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba32ffd700 1 -- 172.21.15.89:0/3577277608 >> v2:172.21.15.172:3300/0 conn(0x7fba340ceda0 msgr2=0x7fba3415c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:53.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba32ffd700 1 --2- 172.21.15.89:0/3577277608 >> v2:172.21.15.172:3300/0 conn(0x7fba340ceda0 0x7fba3415c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba32ffd700 1 -- 172.21.15.89:0/3577277608 >> v1:172.21.15.172:6789/0 conn(0x7fba3406b610 legacy=0x7fba3406c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba32ffd700 1 -- 172.21.15.89:0/3577277608 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba340cd770 con 0x7fba3406bdb0 2023-07-06T02:23:53.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba337fe700 1 --2- 172.21.15.89:0/3577277608 >> v2:172.21.15.172:3300/0 conn(0x7fba340ceda0 0x7fba3415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:53.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba32ffd700 1 --2- 172.21.15.89:0/3577277608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba3406bdb0 0x7fba3406aa70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba28009840 tx=0x7fba28009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20266fbef6820c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:53.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba31ffb700 1 -- 172.21.15.89:0/3577277608 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28011830 con 0x7fba3406bdb0 2023-07-06T02:23:53.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba31ffb700 1 -- 172.21.15.89:0/3577277608 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba28011990 con 0x7fba3406bdb0 2023-07-06T02:23:53.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.774+0000 7fba31ffb700 1 -- 172.21.15.89:0/3577277608 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28011d20 con 0x7fba3406bdb0 2023-07-06T02:23:53.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 -- 172.21.15.89:0/3577277608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba3406bdb0 msgr2=0x7fba3406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 --2- 172.21.15.89:0/3577277608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba3406bdb0 0x7fba3406aa70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba28009840 tx=0x7fba28009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:53.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 -- 172.21.15.89:0/3577277608 shutdown_connections 2023-07-06T02:23:53.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 --2- 172.21.15.89:0/3577277608 >> v2:172.21.15.172:3300/0 conn(0x7fba340ceda0 0x7fba3415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 --2- 172.21.15.89:0/3577277608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba3406bdb0 0x7fba3406aa70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 -- 172.21.15.89:0/3577277608 >> 172.21.15.89:0/3577277608 conn(0x7fba340b5390 msgr2=0x7fba340d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:53.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 -- 172.21.15.89:0/3577277608 shutdown_connections 2023-07-06T02:23:53.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.775+0000 7fba39552700 1 -- 172.21.15.89:0/3577277608 wait complete. 2023-07-06T02:23:53.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.776+0000 7fba39552700 1 Processor -- start 2023-07-06T02:23:53.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.776+0000 7fba39552700 1 -- start start 2023-07-06T02:23:53.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba39552700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba3406bdb0 0x7fba340b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba39552700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba340ceda0 0x7fba340cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba39552700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba34152770 0x7fba340cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba39552700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba340c67a0 con 0x7fba34152770 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba39552700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba340c6a40 con 0x7fba3406bdb0 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba39552700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba340c6d20 con 0x7fba340ceda0 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba34152770 0x7fba340cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba34152770 0x7fba340cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34204/0 (socket says 172.21.15.89:34204) 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 -- 172.21.15.89:0/2710681054 learned_addr learned my addr 172.21.15.89:0/2710681054 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba327fc700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba340ceda0 0x7fba340cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba32ffd700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba3406bdb0 0x7fba340b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 -- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba340ceda0 msgr2=0x7fba340cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba340ceda0 0x7fba340cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 -- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba3406bdb0 msgr2=0x7fba340b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba3406bdb0 0x7fba340b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 -- 172.21.15.89:0/2710681054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c009b30 con 0x7fba34152770 2023-07-06T02:23:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.777+0000 7fba337fe700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba34152770 0x7fba340cc2d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00efc0 tx=0x7fba2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.778+0000 7fba1b7fe700 1 -- 172.21.15.89:0/2710681054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c018070 con 0x7fba34152770 2023-07-06T02:23:53.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.778+0000 7fba1b7fe700 1 -- 172.21.15.89:0/2710681054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c00f040 con 0x7fba34152770 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.778+0000 7fba1b7fe700 1 -- 172.21.15.89:0/2710681054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c003c40 con 0x7fba34152770 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.778+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba280094a0 con 0x7fba34152770 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.778+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba340c72e0 con 0x7fba34152770 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.779+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba34074270 con 0x7fba34152770 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.779+0000 7fba1b7fe700 1 -- 172.21.15.89:0/2710681054 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba2c00c9f0 con 0x7fba34152770 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.779+0000 7fba1b7fe700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fba2006ace0 0x7fba2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.780+0000 7fba32ffd700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fba2006ace0 0x7fba2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:53.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.780+0000 7fba1b7fe700 1 -- 172.21.15.89:0/2710681054 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fba2c089980 con 0x7fba34152770 2023-07-06T02:23:53.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.780+0000 7fba32ffd700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fba2006ace0 0x7fba2006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fba28000c00 tx=0x7fba28008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:53.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.784+0000 7fba1b7fe700 1 -- 172.21.15.89:0/2710681054 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba2c058220 con 0x7fba34152770 2023-07-06T02:23:53.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.972+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba340c74a0 con 0x7fba34152770 2023-07-06T02:23:53.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.974+0000 7fba1b7fe700 1 -- 172.21.15.89:0/2710681054 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fba2c057db0 con 0x7fba34152770 2023-07-06T02:23:53.977 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:53.977 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:23:41.389931+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:53.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fba2006ace0 msgr2=0x7fba2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fba2006ace0 0x7fba2006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fba28000c00 tx=0x7fba28008030 comp rx=0 tx=0).stop 2023-07-06T02:23:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba34152770 msgr2=0x7fba340cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba34152770 0x7fba340cc2d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00efc0 tx=0x7fba2c00c3a0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 shutdown_connections 2023-07-06T02:23:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fba2006ace0 0x7fba2006d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba34152770 0x7fba340cc2d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba3406bdb0 0x7fba340b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 --2- 172.21.15.89:0/2710681054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba340ceda0 0x7fba340cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 >> 172.21.15.89:0/2710681054 conn(0x7fba340b5390 msgr2=0x7fba3406c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 shutdown_connections 2023-07-06T02:23:53.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:53.976+0000 7fba39552700 1 -- 172.21.15.89:0/2710681054 wait complete. 2023-07-06T02:23:53.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:22:50.488557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:22:51.770162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-06T02:22:55.444256+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-06T02:23:53.992 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:23:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.113+0000 7f0aca3f8700 1 Processor -- start 2023-07-06T02:23:54.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.113+0000 7f0aca3f8700 1 -- start start 2023-07-06T02:23:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0aca3f8700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ac406dc10 0x7f0ac415a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0aca3f8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac40cd350 con 0x7f0ac415adb0 2023-07-06T02:23:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0aca3f8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac406e110 con 0x7f0ac40705a0 2023-07-06T02:23:54.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0aca3f8700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac406e3f0 con 0x7f0ac406dc10 2023-07-06T02:23:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ac406dc10 0x7f0ac415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ac406dc10 0x7f0ac415a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34210/0 (socket says 172.21.15.89:34210) 2023-07-06T02:23:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac8995700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0ac415adb0 0x7f0ac4157950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59114/0 (socket says 172.21.15.89:59114) 2023-07-06T02:23:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac37fe700 1 -- 172.21.15.89:0/2767432397 learned_addr learned my addr 172.21.15.89:0/2767432397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac37fe700 1 -- 172.21.15.89:0/2767432397 >> v1:172.21.15.172:6789/0 conn(0x7f0ac40705a0 legacy=0x7f0ac40ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac37fe700 1 -- 172.21.15.89:0/2767432397 >> v1:172.21.15.89:6789/0 conn(0x7f0ac415adb0 legacy=0x7f0ac4157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac2ffd700 1 -- 172.21.15.89:0/2767432397 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843936328 0 0) 0x7f0ac40cd350 con 0x7f0ac415adb0 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac37fe700 1 -- 172.21.15.89:0/2767432397 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac406e6d0 con 0x7f0ac406dc10 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac37fe700 1 --2- 172.21.15.89:0/2767432397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac406dc10 0x7f0ac415a870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009920 tx=0x7f0ab4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712ccfbaa54237be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac2ffd700 1 -- 172.21.15.89:0/2767432397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4004230 con 0x7f0ac406dc10 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac2ffd700 1 -- 172.21.15.89:0/2767432397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4004390 con 0x7f0ac406dc10 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.114+0000 7f0ac2ffd700 1 -- 172.21.15.89:0/2767432397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4009e70 con 0x7f0ac406dc10 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.115+0000 7f0aca3f8700 1 -- 172.21.15.89:0/2767432397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac406dc10 msgr2=0x7f0ac415a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.115+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/2767432397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac406dc10 0x7f0ac415a870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009920 tx=0x7f0ab4009c30 comp rx=0 tx=0).stop 2023-07-06T02:23:54.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.115+0000 7f0aca3f8700 1 -- 172.21.15.89:0/2767432397 shutdown_connections 2023-07-06T02:23:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.115+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/2767432397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac406dc10 0x7f0ac415a870 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.115+0000 7f0aca3f8700 1 -- 172.21.15.89:0/2767432397 >> 172.21.15.89:0/2767432397 conn(0x7f0ac40d0b00 msgr2=0x7f0ac40d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.115+0000 7f0aca3f8700 1 -- 172.21.15.89:0/2767432397 shutdown_connections 2023-07-06T02:23:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.115+0000 7f0aca3f8700 1 -- 172.21.15.89:0/2767432397 wait complete. 2023-07-06T02:23:54.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.116+0000 7f0aca3f8700 1 Processor -- start 2023-07-06T02:23:54.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.116+0000 7f0aca3f8700 1 -- start start 2023-07-06T02:23:54.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0aca3f8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ac406dc10 0x7f0ac41f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0aca3f8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac40705a0 0x7f0ac414ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0aca3f8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ac415adb0 0x7f0ac40c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0aca3f8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41f4130 con 0x7f0ac40705a0 2023-07-06T02:23:54.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0aca3f8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41543b0 con 0x7f0ac415adb0 2023-07-06T02:23:54.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0aca3f8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41537a0 con 0x7f0ac406dc10 2023-07-06T02:23:54.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac40705a0 0x7f0ac414ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac40705a0 0x7f0ac414ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34214/0 (socket says 172.21.15.89:34214) 2023-07-06T02:23:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0ac37fe700 1 -- 172.21.15.89:0/3600255399 learned_addr learned my addr 172.21.15.89:0/3600255399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0ac8995700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ac415adb0 0x7f0ac40c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.117+0000 7f0ac3fff700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ac406dc10 0x7f0ac41f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac37fe700 1 -- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ac406dc10 msgr2=0x7f0ac41f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac37fe700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ac406dc10 0x7f0ac41f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac37fe700 1 -- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ac415adb0 msgr2=0x7f0ac40c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac37fe700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ac415adb0 0x7f0ac40c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac37fe700 1 -- 172.21.15.89:0/3600255399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009550 con 0x7f0ac40705a0 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac8995700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ac415adb0 0x7f0ac40c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac37fe700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac40705a0 0x7f0ac414ee50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009fd0 tx=0x7f0ab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac0ff9700 1 -- 172.21.15.89:0/3600255399 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400be50 con 0x7f0ac40705a0 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac0ff9700 1 -- 172.21.15.89:0/3600255399 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab401f070 con 0x7f0ac40705a0 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac4153a80 con 0x7f0ac40705a0 2023-07-06T02:23:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0ac0ff9700 1 -- 172.21.15.89:0/3600255399 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40038c0 con 0x7f0ac40705a0 2023-07-06T02:23:54.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.118+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac4154040 con 0x7f0ac40705a0 2023-07-06T02:23:54.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.119+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac4154520 con 0x7f0ac40705a0 2023-07-06T02:23:54.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.119+0000 7f0ac0ff9700 1 -- 172.21.15.89:0/3600255399 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ab4021030 con 0x7f0ac40705a0 2023-07-06T02:23:54.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.120+0000 7f0ac0ff9700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0ab006ad10 0x7f0ab006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.120+0000 7f0ac0ff9700 1 -- 172.21.15.89:0/3600255399 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0ab408aeb0 con 0x7f0ac40705a0 2023-07-06T02:23:54.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.120+0000 7f0ac3fff700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0ab006ad10 0x7f0ab006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.120+0000 7f0ac3fff700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0ab006ad10 0x7f0ab006d1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8001670 tx=0x7f0ab8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.124+0000 7f0ac0ff9700 1 -- 172.21.15.89:0/3600255399 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0ab4055de0 con 0x7f0ac40705a0 2023-07-06T02:23:54.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.311+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ac4154520 con 0x7f0ac40705a0 2023-07-06T02:23:54.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.311+0000 7f0ac0ff9700 1 -- 172.21.15.89:0/3600255399 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0ab4059400 con 0x7f0ac40705a0 2023-07-06T02:23:54.315 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:23:54.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0ab006ad10 msgr2=0x7f0ab006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0ab006ad10 0x7f0ab006d1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8001670 tx=0x7f0ab8009040 comp rx=0 tx=0).stop 2023-07-06T02:23:54.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac40705a0 msgr2=0x7f0ac414ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac40705a0 0x7f0ac414ee50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009fd0 tx=0x7f0ab4004750 comp rx=0 tx=0).stop 2023-07-06T02:23:54.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 shutdown_connections 2023-07-06T02:23:54.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ac415adb0 0x7f0ac40c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ac406dc10 0x7f0ac41f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0ab006ad10 0x7f0ab006d1c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 --2- 172.21.15.89:0/3600255399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ac40705a0 0x7f0ac414ee50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 >> 172.21.15.89:0/3600255399 conn(0x7f0ac40d0b00 msgr2=0x7f0ac4070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:54.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 shutdown_connections 2023-07-06T02:23:54.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.313+0000 7f0aca3f8700 1 -- 172.21.15.89:0/3600255399 wait complete. 2023-07-06T02:23:54.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:23:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.449+0000 7f0ddb087700 1 Processor -- start 2023-07-06T02:23:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.449+0000 7f0ddb087700 1 -- start start 2023-07-06T02:23:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0ddb087700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0dd415c500 0x7f0dd406edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0ddb087700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0ddb087700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd40cb9f0 con 0x7f0dd4070f20 2023-07-06T02:23:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0ddb087700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd40718e0 con 0x7f0dd415c500 2023-07-06T02:23:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0ddb087700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd415d8e0 con 0x7f0dd4160710 2023-07-06T02:23:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd3fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0dd4070f20 0x7f0dd4160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59126/0 (socket says 172.21.15.89:59126) 2023-07-06T02:23:54.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd3fff700 1 -- 172.21.15.89:0/844934236 learned_addr learned my addr 172.21.15.89:0/844934236 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:54.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd8e23700 1 --2- 172.21.15.89:0/844934236 >> v2:172.21.15.172:3300/0 conn(0x7f0dd415c500 0x7f0dd406edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd9624700 1 --2- 172.21.15.89:0/844934236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd37fe700 1 -- 172.21.15.89:0/844934236 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574035317 0 0) 0x7f0dd40cb9f0 con 0x7f0dd4070f20 2023-07-06T02:23:54.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd37fe700 1 -- 172.21.15.89:0/844934236 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc40035b0 con 0x7f0dd4070f20 2023-07-06T02:23:54.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd37fe700 1 -- 172.21.15.89:0/844934236 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624310394 0 0) 0x7f0dc40035b0 con 0x7f0dd4070f20 2023-07-06T02:23:54.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd9624700 1 -- 172.21.15.89:0/844934236 >> v2:172.21.15.172:3300/0 conn(0x7f0dd415c500 msgr2=0x7f0dd406edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd9624700 1 --2- 172.21.15.89:0/844934236 >> v2:172.21.15.172:3300/0 conn(0x7f0dd415c500 0x7f0dd406edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd9624700 1 -- 172.21.15.89:0/844934236 >> v1:172.21.15.89:6789/0 conn(0x7f0dd4070f20 legacy=0x7f0dd4160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd9624700 1 -- 172.21.15.89:0/844934236 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd406f310 con 0x7f0dd4160710 2023-07-06T02:23:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.450+0000 7f0dd8e23700 1 --2- 172.21.15.89:0/844934236 >> v2:172.21.15.172:3300/0 conn(0x7f0dd415c500 0x7f0dd406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:23:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.451+0000 7f0dd9624700 1 --2- 172.21.15.89:0/844934236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40713a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009d90 tx=0x7f0dcc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a100d80ffc3001 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.451+0000 7f0dd37fe700 1 -- 172.21.15.89:0/844934236 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00e070 con 0x7f0dd4160710 2023-07-06T02:23:54.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.451+0000 7f0dd37fe700 1 -- 172.21.15.89:0/844934236 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc003a20 con 0x7f0dd4160710 2023-07-06T02:23:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.451+0000 7f0dd37fe700 1 -- 172.21.15.89:0/844934236 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0001670 con 0x7f0dd4160710 2023-07-06T02:23:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 -- 172.21.15.89:0/844934236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 msgr2=0x7f0dd40713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 --2- 172.21.15.89:0/844934236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40713a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009d90 tx=0x7f0dcc00b330 comp rx=0 tx=0).stop 2023-07-06T02:23:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 -- 172.21.15.89:0/844934236 shutdown_connections 2023-07-06T02:23:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 --2- 172.21.15.89:0/844934236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40713a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 --2- 172.21.15.89:0/844934236 >> v2:172.21.15.172:3300/0 conn(0x7f0dd415c500 0x7f0dd406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 -- 172.21.15.89:0/844934236 >> 172.21.15.89:0/844934236 conn(0x7f0dd40cf1c0 msgr2=0x7f0dd40cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 -- 172.21.15.89:0/844934236 shutdown_connections 2023-07-06T02:23:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.452+0000 7f0ddb087700 1 -- 172.21.15.89:0/844934236 wait complete. 2023-07-06T02:23:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.453+0000 7f0ddb087700 1 Processor -- start 2023-07-06T02:23:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.453+0000 7f0ddb087700 1 -- start start 2023-07-06T02:23:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0ddb087700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0dd4070f20 0x7f0dd40c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0ddb087700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dd415c500 0x7f0dd4158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0ddb087700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0ddb087700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40b19c0 con 0x7f0dd415c500 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0ddb087700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40c8e70 con 0x7f0dd4070f20 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0ddb087700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40b34a0 con 0x7f0dd4160710 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dd415c500 0x7f0dd4158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dd415c500 0x7f0dd4158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34230/0 (socket says 172.21.15.89:34230) 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 -- 172.21.15.89:0/1870004973 learned_addr learned my addr 172.21.15.89:0/1870004973 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd8e23700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0dd4070f20 0x7f0dd40c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd9624700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 -- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 msgr2=0x7f0dd40ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 -- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0dd4070f20 msgr2=0x7f0dd40c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0dd4070f20 0x7f0dd40c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.454+0000 7f0dd3fff700 1 -- 172.21.15.89:0/1870004973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80094a0 con 0x7f0dd415c500 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.455+0000 7f0dd3fff700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dd415c500 0x7f0dd4158ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00011e0 tx=0x7f0dc000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.455+0000 7f0dd17fa700 1 -- 172.21.15.89:0/1870004973 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0004c80 con 0x7f0dd415c500 2023-07-06T02:23:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.455+0000 7f0dd17fa700 1 -- 172.21.15.89:0/1870004973 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc0004de0 con 0x7f0dd415c500 2023-07-06T02:23:54.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.455+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc009a70 con 0x7f0dd415c500 2023-07-06T02:23:54.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.455+0000 7f0dd17fa700 1 -- 172.21.15.89:0/1870004973 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00072d0 con 0x7f0dd415c500 2023-07-06T02:23:54.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.455+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd40b3760 con 0x7f0dd415c500 2023-07-06T02:23:54.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.456+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4074270 con 0x7f0dd415c500 2023-07-06T02:23:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.456+0000 7f0dd17fa700 1 -- 172.21.15.89:0/1870004973 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0dc0007430 con 0x7f0dd415c500 2023-07-06T02:23:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.456+0000 7f0dd17fa700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0db806ace0 0x7f0db806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.457+0000 7f0dd17fa700 1 -- 172.21.15.89:0/1870004973 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0dc0089510 con 0x7f0dd415c500 2023-07-06T02:23:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.457+0000 7f0dd8e23700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0db806ace0 0x7f0db806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.457+0000 7f0dd8e23700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0db806ace0 0x7f0db806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009d70 tx=0x7f0dc8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.461+0000 7f0dd17fa700 1 -- 172.21.15.89:0/1870004973 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0dc0054320 con 0x7f0dd415c500 2023-07-06T02:23:54.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.646+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0dd4071b10 con 0x7f0dd415c500 2023-07-06T02:23:54.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.646+0000 7f0dd17fa700 1 -- 172.21.15.89:0/1870004973 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0dc0057940 con 0x7f0dd415c500 2023-07-06T02:23:54.650 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:23:54.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0db806ace0 msgr2=0x7f0db806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0db806ace0 0x7f0db806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009d70 tx=0x7f0dc8009210 comp rx=0 tx=0).stop 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dd415c500 msgr2=0x7f0dd4158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dd415c500 0x7f0dd4158ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00011e0 tx=0x7f0dc000ca50 comp rx=0 tx=0).stop 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 shutdown_connections 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0dd4160710 0x7f0dd40ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0dd4070f20 0x7f0dd40c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0db806ace0 0x7f0db806d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 --2- 172.21.15.89:0/1870004973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dd415c500 0x7f0dd4158ef0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 >> 172.21.15.89:0/1870004973 conn(0x7f0dd40cf1c0 msgr2=0x7f0dd415c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:54.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 shutdown_connections 2023-07-06T02:23:54.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.648+0000 7f0ddb087700 1 -- 172.21.15.89:0/1870004973 wait complete. 2023-07-06T02:23:54.663 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:23:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.783+0000 7f05379d8700 1 Processor -- start 2023-07-06T02:23:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.783+0000 7f05379d8700 1 -- start start 2023-07-06T02:23:54.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f05379d8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05301600b0 0x7f05300b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f05379d8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300cb3d0 con 0x7f05300b19d0 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f05379d8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300b1dd0 con 0x7f0530156ab0 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f05379d8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0530156e80 con 0x7f05301600b0 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f0535f75700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f05300b19d0 0x7f05300cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59142/0 (socket says 172.21.15.89:59142) 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f0535f75700 1 -- 172.21.15.89:0/4229665387 learned_addr learned my addr 172.21.15.89:0/4229665387 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f0534f73700 1 --2- 172.21.15.89:0/4229665387 >> v2:172.21.15.172:3300/0 conn(0x7f05301600b0 0x7f05300b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758879189 0 0) 0x7f05300cb3d0 con 0x7f05300b19d0 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.784+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c0035b0 con 0x7f05300b19d0 2023-07-06T02:23:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805087078 0 0) 0x7f051c0035b0 con 0x7f05300b19d0 2023-07-06T02:23:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 >> v2:172.21.15.172:3300/0 conn(0x7f05301600b0 msgr2=0x7f05300b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 --2- 172.21.15.89:0/4229665387 >> v2:172.21.15.172:3300/0 conn(0x7f05301600b0 0x7f05300b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 >> v1:172.21.15.172:6789/0 conn(0x7f0530156ab0 legacy=0x7f0530160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300ca370 con 0x7f05300b19d0 2023-07-06T02:23:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f05280019f0 con 0x7f05300b19d0 2023-07-06T02:23:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 >> v1:172.21.15.89:6789/0 conn(0x7f05300b19d0 legacy=0x7f05300cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f051c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f051c0043f0 0x7f051c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f051c0050b0 0x7f051c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530156e80 con 0x7f051c003a00 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00c000 con 0x7f051c0050b0 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00c1c0 con 0x7f051c0043f0 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535774700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f051c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0534f73700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f051c0043f0 0x7f051c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535f75700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f051c0050b0 0x7f051c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535774700 1 -- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f051c0043f0 msgr2=0x7f051c0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535774700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f051c0043f0 0x7f051c0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535774700 1 -- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f051c0050b0 msgr2=0x7f051c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535774700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f051c0050b0 0x7f051c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535774700 1 -- 172.21.15.89:0/4229665387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0035b0 con 0x7f051c003a00 2023-07-06T02:23:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535f75700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f051c0050b0 0x7f051c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0534f73700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f051c0043f0 0x7f051c0048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535f75700 1 -- 172.21.15.89:0/4229665387 reap_dead start 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0535774700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f051c003dd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f05200018d0 tx=0x7f051c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7a4cf6cebd5866 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.785+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200052d0 con 0x7f051c003a00 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520005430 con 0x7f051c003a00 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f0527fff700 1 -- 172.21.15.89:0/4229665387 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200038c0 con 0x7f051c003a00 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f05379d8700 1 -- 172.21.15.89:0/4229665387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 msgr2=0x7f051c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f05379d8700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f051c003dd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f05200018d0 tx=0x7f051c005880 comp rx=0 tx=0).stop 2023-07-06T02:23:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f05379d8700 1 -- 172.21.15.89:0/4229665387 shutdown_connections 2023-07-06T02:23:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f05379d8700 1 --2- 172.21.15.89:0/4229665387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f051c003dd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f05379d8700 1 -- 172.21.15.89:0/4229665387 >> 172.21.15.89:0/4229665387 conn(0x7f05300cc490 msgr2=0x7f05300cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f05379d8700 1 -- 172.21.15.89:0/4229665387 shutdown_connections 2023-07-06T02:23:54.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.786+0000 7f05379d8700 1 -- 172.21.15.89:0/4229665387 wait complete. 2023-07-06T02:23:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.788+0000 7f05379d8700 1 Processor -- start 2023-07-06T02:23:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.788+0000 7f05379d8700 1 -- start start 2023-07-06T02:23:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f05379d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f05300c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f05379d8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05300c5cc0 0x7f05300c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f05379d8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05300c7020 0x7f05300c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f05379d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530145b00 con 0x7f051c003a00 2023-07-06T02:23:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f05379d8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300c74a0 con 0x7f05300c7020 2023-07-06T02:23:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f05379d8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300c4c20 con 0x7f05300c5cc0 2023-07-06T02:23:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f05300c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f05300c4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34248/0 (socket says 172.21.15.89:34248) 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 -- 172.21.15.89:0/3218849536 learned_addr learned my addr 172.21.15.89:0/3218849536 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535f75700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05300c7020 0x7f05300c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0534f73700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05300c5cc0 0x7f05300c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 -- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05300c5cc0 msgr2=0x7f05300c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05300c5cc0 0x7f05300c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 -- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05300c7020 msgr2=0x7f05300c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05300c7020 0x7f05300c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 -- 172.21.15.89:0/3218849536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300c4f00 con 0x7f051c003a00 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535f75700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05300c7020 0x7f05300c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0535774700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f05300c4590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f051c005810 tx=0x7f0520001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0525ffb700 1 -- 172.21.15.89:0/3218849536 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052000c690 con 0x7f051c003a00 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0525ffb700 1 -- 172.21.15.89:0/3218849536 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052000c7f0 con 0x7f051c003a00 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05300c51e0 con 0x7f051c003a00 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.789+0000 7f0525ffb700 1 -- 172.21.15.89:0/3218849536 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052000cba0 con 0x7f051c003a00 2023-07-06T02:23:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.790+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05300c57a0 con 0x7f051c003a00 2023-07-06T02:23:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.790+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05300c5960 con 0x7f051c003a00 2023-07-06T02:23:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.791+0000 7f0525ffb700 1 -- 172.21.15.89:0/3218849536 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f052000cd00 con 0x7f051c003a00 2023-07-06T02:23:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.791+0000 7f0525ffb700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f051406ace0 0x7f051406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.791+0000 7f0525ffb700 1 -- 172.21.15.89:0/3218849536 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f052008a5f0 con 0x7f051c003a00 2023-07-06T02:23:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.791+0000 7f0534f73700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f051406ace0 0x7f051406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.792+0000 7f0534f73700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f051406ace0 0x7f051406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05300c3370 tx=0x7f052c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.796+0000 7f0525ffb700 1 -- 172.21.15.89:0/3218849536 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0520055380 con 0x7f051c003a00 2023-07-06T02:23:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.981+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0530074270 con 0x7f051c003a00 2023-07-06T02:23:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.981+0000 7f0525ffb700 1 -- 172.21.15.89:0/3218849536 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0520019310 con 0x7f051c003a00 2023-07-06T02:23:54.985 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.983+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f051406ace0 msgr2=0x7f051406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.983+0000 7f05379d8700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f051406ace0 0x7f051406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05300c3370 tx=0x7f052c008040 comp rx=0 tx=0).stop 2023-07-06T02:23:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.983+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 msgr2=0x7f05300c4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.983+0000 7f05379d8700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f05300c4590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f051c005810 tx=0x7f0520001230 comp rx=0 tx=0).stop 2023-07-06T02:23:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.983+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 shutdown_connections 2023-07-06T02:23:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.984+0000 7f05379d8700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05300c7020 0x7f05300c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.984+0000 7f05379d8700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f051406ace0 0x7f051406d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.984+0000 7f05379d8700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c003a00 0x7f05300c4590 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.984+0000 7f05379d8700 1 --2- 172.21.15.89:0/3218849536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05300c5cc0 0x7f05300c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.984+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 >> 172.21.15.89:0/3218849536 conn(0x7f05300cc490 msgr2=0x7f05300cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.984+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 shutdown_connections 2023-07-06T02:23:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:54.984+0000 7f05379d8700 1 -- 172.21.15.89:0/3218849536 wait complete. 2023-07-06T02:23:54.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:23:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.119+0000 7fcc2152c700 1 Processor -- start 2023-07-06T02:23:55.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.119+0000 7fcc2152c700 1 -- start start 2023-07-06T02:23:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc2152c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc1c158960 0x7fcc1c15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc2152c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc1c0ccea0 0x7fcc1c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc2152c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c0c9520 0x7fcc1c070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc2152c700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1c15d390 con 0x7fcc1c158960 2023-07-06T02:23:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc2152c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1c06c770 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc2152c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c06d5d0 con 0x7fcc1c0c9520 2023-07-06T02:23:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc1c158960 0x7fcc1c15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcc1c158960 0x7fcc1c15ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34256/0 (socket says 172.21.15.89:34256) 2023-07-06T02:23:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 -- 172.21.15.89:0/365485119 learned_addr learned my addr 172.21.15.89:0/365485119 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1a7fc700 1 --2- 172.21.15.89:0/365485119 >> v2:172.21.15.172:3300/0 conn(0x7fcc1c0ccea0 0x7fcc1c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 -- 172.21.15.89:0/365485119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c0c9520 msgr2=0x7fcc1c070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 --2- 172.21.15.89:0/365485119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c0c9520 0x7fcc1c070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 -- 172.21.15.89:0/365485119 >> v2:172.21.15.172:3300/0 conn(0x7fcc1c0ccea0 msgr2=0x7fcc1c0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 --2- 172.21.15.89:0/365485119 >> v2:172.21.15.172:3300/0 conn(0x7fcc1c0ccea0 0x7fcc1c0c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 -- 172.21.15.89:0/365485119 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c158d30 con 0x7fcc1c158960 2023-07-06T02:23:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.120+0000 7fcc1affd700 1 --2- 172.21.15.89:0/365485119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c158960 0x7fcc1c15ce50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c06ea90 tx=0x7fcc0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6727fc7e47c5287a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc19ffb700 1 -- 172.21.15.89:0/365485119 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00b870 con 0x7fcc1c158960 2023-07-06T02:23:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc19ffb700 1 -- 172.21.15.89:0/365485119 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c00b9d0 con 0x7fcc1c158960 2023-07-06T02:23:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc19ffb700 1 -- 172.21.15.89:0/365485119 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00bd60 con 0x7fcc1c158960 2023-07-06T02:23:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 -- 172.21.15.89:0/365485119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c158960 msgr2=0x7fcc1c15ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 --2- 172.21.15.89:0/365485119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c158960 0x7fcc1c15ce50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c06ea90 tx=0x7fcc0c009bd0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 -- 172.21.15.89:0/365485119 shutdown_connections 2023-07-06T02:23:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 --2- 172.21.15.89:0/365485119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c0c9520 0x7fcc1c070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 --2- 172.21.15.89:0/365485119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c158960 0x7fcc1c15ce50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 --2- 172.21.15.89:0/365485119 >> v2:172.21.15.172:3300/0 conn(0x7fcc1c0ccea0 0x7fcc1c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 -- 172.21.15.89:0/365485119 >> 172.21.15.89:0/365485119 conn(0x7fcc1c0ce870 msgr2=0x7fcc1c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 -- 172.21.15.89:0/365485119 shutdown_connections 2023-07-06T02:23:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.121+0000 7fcc2152c700 1 -- 172.21.15.89:0/365485119 wait complete. 2023-07-06T02:23:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.122+0000 7fcc2152c700 1 Processor -- start 2023-07-06T02:23:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc2152c700 1 -- start start 2023-07-06T02:23:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc2152c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc1c0c9520 0x7fcc1c0c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc2152c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c0ccea0 0x7fcc1c0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc2152c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c158960 0x7fcc1c1efc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc2152c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0c72f0 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc2152c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0c7010 con 0x7fcc1c0c9520 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc2152c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0c24d0 con 0x7fcc1c158960 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c0ccea0 0x7fcc1c0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc1a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c0ccea0 0x7fcc1c0c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34264/0 (socket says 172.21.15.89:34264) 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.123+0000 7fcc1a7fc700 1 -- 172.21.15.89:0/3223484055 learned_addr learned my addr 172.21.15.89:0/3223484055 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:55.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1affd700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc1c0c9520 0x7fcc1c0c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1b7fe700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c158960 0x7fcc1c1efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1a7fc700 1 -- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c158960 msgr2=0x7fcc1c1efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1a7fc700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c158960 0x7fcc1c1efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1a7fc700 1 -- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc1c0c9520 msgr2=0x7fcc1c0c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1a7fc700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc1c0c9520 0x7fcc1c0c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1a7fc700 1 -- 172.21.15.89:0/3223484055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c009550 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc1a7fc700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c0ccea0 0x7fcc1c0c67c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c7f0 tx=0x7fcc1000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc037fe700 1 -- 172.21.15.89:0/3223484055 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10011800 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc037fe700 1 -- 172.21.15.89:0/3223484055 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10011960 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c0c27b0 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc037fe700 1 -- 172.21.15.89:0/3223484055 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10011cf0 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.124+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c0c6d00 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.125+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1c0c2cd0 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.125+0000 7fcc037fe700 1 -- 172.21.15.89:0/3223484055 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc10018360 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.126+0000 7fcc037fe700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcc0806ace0 0x7fcc0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.126+0000 7fcc037fe700 1 -- 172.21.15.89:0/3223484055 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcc10089760 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.126+0000 7fcc1affd700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcc0806ace0 0x7fcc0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.126+0000 7fcc1affd700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcc0806ace0 0x7fcc0806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c014040 tx=0x7fcc0c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.130+0000 7fcc037fe700 1 -- 172.21.15.89:0/3223484055 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc1c0c2cd0 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.316+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc1c0c2cd0 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.316+0000 7fcc037fe700 1 -- 172.21.15.89:0/3223484055 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc1c0c2cd0 con 0x7fcc1c0ccea0 2023-07-06T02:23:55.320 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:23:55.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcc0806ace0 msgr2=0x7fcc0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcc0806ace0 0x7fcc0806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c014040 tx=0x7fcc0c008030 comp rx=0 tx=0).stop 2023-07-06T02:23:55.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c0ccea0 msgr2=0x7fcc1c0c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c0ccea0 0x7fcc1c0c67c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c7f0 tx=0x7fcc1000cbb0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 shutdown_connections 2023-07-06T02:23:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1c158960 0x7fcc1c1efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc1c0c9520 0x7fcc1c0c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcc0806ace0 0x7fcc0806d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 --2- 172.21.15.89:0/3223484055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1c0ccea0 0x7fcc1c0c67c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 >> 172.21.15.89:0/3223484055 conn(0x7fcc1c0ce870 msgr2=0x7fcc1c0ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:55.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 shutdown_connections 2023-07-06T02:23:55.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.318+0000 7fcc2152c700 1 -- 172.21.15.89:0/3223484055 wait complete. 2023-07-06T02:23:55.333 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-06T02:23:55.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.454+0000 7fe0e4541700 1 Processor -- start 2023-07-06T02:23:55.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.454+0000 7fe0e4541700 1 -- start start 2023-07-06T02:23:55.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e4541700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc0cecf0 0x7fe0dc0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e4541700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe0dc0ca5f0 0x7fe0dc06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e4541700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc06b160 con 0x7fe0dc06e250 2023-07-06T02:23:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e4541700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc06abe0 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e4541700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc06ec00 con 0x7fe0dc0cecf0 2023-07-06T02:23:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e2ade700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe0dc0ca5f0 0x7fe0dc06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e1adc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe0dc06e250 0x7fe0dc0c9ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59174/0 (socket says 172.21.15.89:59174) 2023-07-06T02:23:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e1adc700 1 -- 172.21.15.89:0/3900301606 learned_addr learned my addr 172.21.15.89:0/3900301606 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e2ade700 1 -- 172.21.15.89:0/3900301606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc0cecf0 msgr2=0x7fe0dc0ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e2ade700 1 --2- 172.21.15.89:0/3900301606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc0cecf0 0x7fe0dc0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e2ade700 1 -- 172.21.15.89:0/3900301606 >> v1:172.21.15.89:6789/0 conn(0x7fe0dc06e250 legacy=0x7fe0dc0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e12db700 1 -- 172.21.15.89:0/3900301606 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827032629 0 0) 0x7fe0dc06b160 con 0x7fe0dc06e250 2023-07-06T02:23:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.455+0000 7fe0e2ade700 1 -- 172.21.15.89:0/3900301606 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0cac10 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e2ade700 1 --2- 172.21.15.89:0/3900301606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc06e6c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009d90 tx=0x7fe0d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889345b957422a1c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e12db700 1 -- 172.21.15.89:0/3900301606 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400e070 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e12db700 1 -- 172.21.15.89:0/3900301606 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d400bbd0 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e12db700 1 -- 172.21.15.89:0/3900301606 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4012430 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 -- 172.21.15.89:0/3900301606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 msgr2=0x7fe0dc06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3900301606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc06e6c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009d90 tx=0x7fe0d400b330 comp rx=0 tx=0).stop 2023-07-06T02:23:55.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 -- 172.21.15.89:0/3900301606 shutdown_connections 2023-07-06T02:23:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3900301606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc06e6c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3900301606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc0cecf0 0x7fe0dc0ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 -- 172.21.15.89:0/3900301606 >> 172.21.15.89:0/3900301606 conn(0x7fe0dc0cf1c0 msgr2=0x7fe0dc0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 -- 172.21.15.89:0/3900301606 shutdown_connections 2023-07-06T02:23:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.456+0000 7fe0e4541700 1 -- 172.21.15.89:0/3900301606 wait complete. 2023-07-06T02:23:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.458+0000 7fe0e4541700 1 Processor -- start 2023-07-06T02:23:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.458+0000 7fe0e4541700 1 -- start start 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e4541700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc06e250 0x7fe0dc0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e4541700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc0a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e4541700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0dc0cecf0 0x7fe0dc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e4541700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1e17d0 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e4541700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1e1a70 con 0x7fe0dc0cecf0 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e4541700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1e1d50 con 0x7fe0dc06e250 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc0a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc0a74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34278/0 (socket says 172.21.15.89:34278) 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 -- 172.21.15.89:0/3467312002 learned_addr learned my addr 172.21.15.89:0/3467312002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e2ade700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0dc0cecf0 0x7fe0dc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e22dd700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc06e250 0x7fe0dc0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 -- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc06e250 msgr2=0x7fe0dc0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc06e250 0x7fe0dc0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 -- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0dc0cecf0 msgr2=0x7fe0dc071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0dc0cecf0 0x7fe0dc071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e1adc700 1 -- 172.21.15.89:0/3467312002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4009a70 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e22dd700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc06e250 0x7fe0dc0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.459+0000 7fe0e2ade700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0dc0cecf0 0x7fe0dc071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.460+0000 7fe0e1adc700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc0a74e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80011e0 tx=0x7fe0d800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.460+0000 7fe0d2ffd700 1 -- 172.21.15.89:0/3467312002 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8004c80 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.460+0000 7fe0d2ffd700 1 -- 172.21.15.89:0/3467312002 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d8004e00 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.460+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc1e2030 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.460+0000 7fe0d2ffd700 1 -- 172.21.15.89:0/3467312002 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8007310 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.460+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc1e25f0 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.461+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc1e27e0 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.461+0000 7fe0d2ffd700 1 -- 172.21.15.89:0/3467312002 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe0d8007470 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.461+0000 7fe0d2ffd700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0c006ace0 0x7fe0c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.461+0000 7fe0d2ffd700 1 -- 172.21.15.89:0/3467312002 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe0d8089520 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.461+0000 7fe0e22dd700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0c006ace0 0x7fe0c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.462+0000 7fe0e22dd700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0c006ace0 0x7fe0c006d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006fd0 tx=0x7fe0cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.466+0000 7fe0d2ffd700 1 -- 172.21.15.89:0/3467312002 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe0d80542b0 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.651+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe0dc074270 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.652+0000 7fe0d2ffd700 1 -- 172.21.15.89:0/3467312002 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe0d80578d0 con 0x7fe0dc0ca5f0 2023-07-06T02:23:55.655 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:23:55.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0c006ace0 msgr2=0x7fe0c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0c006ace0 0x7fe0c006d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006fd0 tx=0x7fe0cc008040 comp rx=0 tx=0).stop 2023-07-06T02:23:55.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 msgr2=0x7fe0dc0a74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc0a74e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80011e0 tx=0x7fe0d800cae0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 shutdown_connections 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0dc0cecf0 0x7fe0dc071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0dc06e250 0x7fe0dc0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe0c006ace0 0x7fe0c006d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 --2- 172.21.15.89:0/3467312002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0dc0ca5f0 0x7fe0dc0a74e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 >> 172.21.15.89:0/3467312002 conn(0x7fe0dc0cf1c0 msgr2=0x7fe0dc0cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:55.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 shutdown_connections 2023-07-06T02:23:55.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.654+0000 7fe0e4541700 1 -- 172.21.15.89:0/3467312002 wait complete. 2023-07-06T02:23:55.669 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:55.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.785+0000 7fbcbd2f8700 1 Processor -- start 2023-07-06T02:23:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.786+0000 7fbcbd2f8700 1 -- start start 2023-07-06T02:23:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.786+0000 7fbcbd2f8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 0x7fbcb80cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.786+0000 7fbcbd2f8700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb815a490 con 0x7fbcb8070a40 2023-07-06T02:23:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.786+0000 7fbcbd2f8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb80d04b0 con 0x7fbcb8160a00 2023-07-06T02:23:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.786+0000 7fbcbd2f8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb815c9b0 con 0x7fbcb815a0c0 2023-07-06T02:23:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.786+0000 7fbcb77fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbcb8070a40 0x7fbcb80b5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59190/0 (socket says 172.21.15.89:59190) 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.786+0000 7fbcb77fe700 1 -- 172.21.15.89:0/186648748 learned_addr learned my addr 172.21.15.89:0/186648748 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb6ffd700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 0x7fbcb80cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129224124 0 0) 0x7fbcb815a490 con 0x7fbcb8070a40 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca00035b0 con 0x7fbcb8070a40 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557745639 0 0) 0x7fbca00035b0 con 0x7fbcb8070a40 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 msgr2=0x7fbcb80cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 0x7fbcb80cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 >> v1:172.21.15.172:6789/0 conn(0x7fbcb8160a00 legacy=0x7fbcb8070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb815d9d0 con 0x7fbcb8070a40 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fbcb00019f0 con 0x7fbcb8070a40 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 >> v1:172.21.15.89:6789/0 conn(0x7fbcb8070a40 legacy=0x7fbcb80b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 0x7fbca0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbca0003fb0 0x7fbca0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb815c9b0 con 0x7fbca0003fb0 2023-07-06T02:23:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca000baf0 con 0x7fbca0004d90 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca000bcb0 con 0x7fbcb815a0c0 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb67fc700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbca0003fb0 0x7fbca0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb6ffd700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 0x7fbca0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb77fe700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb67fc700 1 -- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 msgr2=0x7fbca0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb67fc700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 0x7fbca0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb67fc700 1 -- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca0004d90 msgr2=0x7fbca0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb67fc700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.787+0000 7fbcb67fc700 1 -- 172.21.15.89:0/186648748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca00035b0 con 0x7fbca0003fb0 2023-07-06T02:23:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcb6ffd700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcb815a0c0 0x7fbca0003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcb77fe700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcb67fc700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbca0003fb0 0x7fbca0004850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbca8007fd0 tx=0x7fbca0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b00fe8bf098ba5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80052d0 con 0x7fbca0003fb0 2023-07-06T02:23:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8005470 con 0x7fbca0003fb0 2023-07-06T02:23:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcb5ffb700 1 -- 172.21.15.89:0/186648748 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80038c0 con 0x7fbca0003fb0 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/186648748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbca0003fb0 msgr2=0x7fbca0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcbd2f8700 1 --2- 172.21.15.89:0/186648748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbca0003fb0 0x7fbca0004850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbca8007fd0 tx=0x7fbca0005880 comp rx=0 tx=0).stop 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcb77fe700 1 -- 172.21.15.89:0/186648748 reap_dead start 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/186648748 shutdown_connections 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/186648748 >> 172.21.15.89:0/186648748 conn(0x7fbcb80d0b00 msgr2=0x7fbcb80d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/186648748 shutdown_connections 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.788+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/186648748 wait complete. 2023-07-06T02:23:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 Processor -- start 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 -- start start 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbca0003fb0 0x7fbcb806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb806bac0 0x7fbcb81556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb80ac790 0x7fbcb80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb806c9a0 con 0x7fbcb806bac0 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80adc20 con 0x7fbcb80ac790 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.790+0000 7fbcbd2f8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80adf00 con 0x7fbca0003fb0 2023-07-06T02:23:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb806bac0 0x7fbcb81556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb806bac0 0x7fbcb81556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34298/0 (socket says 172.21.15.89:34298) 2023-07-06T02:23:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 -- 172.21.15.89:0/1843891577 learned_addr learned my addr 172.21.15.89:0/1843891577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb6ffd700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbca0003fb0 0x7fbcb806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb77fe700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb80ac790 0x7fbcb80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 -- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbca0003fb0 msgr2=0x7fbcb806d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbca0003fb0 0x7fbcb806d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 -- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb80ac790 msgr2=0x7fbcb80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb80ac790 0x7fbcb80aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 -- 172.21.15.89:0/1843891577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80c55c0 con 0x7fbcb806bac0 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb77fe700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb80ac790 0x7fbcb80aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcb67fc700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb806bac0 0x7fbcb81556c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbca8004000 tx=0x7fbca8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbc9f7fe700 1 -- 172.21.15.89:0/1843891577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8006a60 con 0x7fbcb806bac0 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbc9f7fe700 1 -- 172.21.15.89:0/1843891577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8006bc0 con 0x7fbcb806bac0 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb80c58a0 con 0x7fbcb806bac0 2023-07-06T02:23:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbc9f7fe700 1 -- 172.21.15.89:0/1843891577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8007030 con 0x7fbcb806bac0 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.791+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb80ad680 con 0x7fbcb806bac0 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.792+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb80c5ef0 con 0x7fbcb806bac0 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.793+0000 7fbc9f7fe700 1 -- 172.21.15.89:0/1843891577 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbca8007190 con 0x7fbcb806bac0 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.793+0000 7fbc9f7fe700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbca406ace0 0x7fbca406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.793+0000 7fbc9f7fe700 1 -- 172.21.15.89:0/1843891577 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbca808ad10 con 0x7fbcb806bac0 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.793+0000 7fbcb6ffd700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbca406ace0 0x7fbca406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.793+0000 7fbcb6ffd700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbca406ace0 0x7fbca406d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009b30 tx=0x7fbcac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:55.797+0000 7fbc9f7fe700 1 -- 172.21.15.89:0/1843891577 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbca8055b20 con 0x7fbcb806bac0 2023-07-06T02:23:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.029+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcb8074270 con 0x7fbcb806bac0 2023-07-06T02:23:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.029+0000 7fbc9f7fe700 1 -- 172.21.15.89:0/1843891577 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fbca8022090 con 0x7fbcb806bac0 2023-07-06T02:23:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:56.033 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:56.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbca406ace0 msgr2=0x7fbca406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbca406ace0 0x7fbca406d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009b30 tx=0x7fbcac008040 comp rx=0 tx=0).stop 2023-07-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb806bac0 msgr2=0x7fbcb81556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb806bac0 0x7fbcb81556c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbca8004000 tx=0x7fbca8001120 comp rx=0 tx=0).stop 2023-07-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 shutdown_connections 2023-07-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb80ac790 0x7fbcb80aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbca0003fb0 0x7fbcb806d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fbca406ace0 0x7fbca406d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 --2- 172.21.15.89:0/1843891577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcb806bac0 0x7fbcb81556c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 >> 172.21.15.89:0/1843891577 conn(0x7fbcb80d0b00 msgr2=0x7fbcb80b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.031+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 shutdown_connections 2023-07-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.032+0000 7fbcbd2f8700 1 -- 172.21.15.89:0/1843891577 wait complete. 2023-07-06T02:23:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:56.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.164+0000 7f56a17ed700 1 Processor -- start 2023-07-06T02:23:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.164+0000 7f56a17ed700 1 -- start start 2023-07-06T02:23:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f56a17ed700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f569c070580 0x7f569c070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f56a17ed700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f569c0ccd40 0x7f569c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f56a17ed700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c070e90 con 0x7f569c0cc7f0 2023-07-06T02:23:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f56a17ed700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c06b2a0 con 0x7f569c0ccd40 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f56a17ed700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c06b580 con 0x7f569c070580 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f569c0ccd40 0x7f569c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569a7fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f569c0cc7f0 0x7f569c06c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59208/0 (socket says 172.21.15.89:59208) 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569a7fc700 1 -- 172.21.15.89:0/1738108663 learned_addr learned my addr 172.21.15.89:0/1738108663 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569b7fe700 1 -- 172.21.15.89:0/1738108663 >> v2:172.21.15.172:3300/0 conn(0x7f569c070580 msgr2=0x7f569c070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569b7fe700 1 --2- 172.21.15.89:0/1738108663 >> v2:172.21.15.172:3300/0 conn(0x7f569c070580 0x7f569c070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569b7fe700 1 -- 172.21.15.89:0/1738108663 >> v1:172.21.15.89:6789/0 conn(0x7f569c0cc7f0 legacy=0x7f569c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569b7fe700 1 -- 172.21.15.89:0/1738108663 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c06b860 con 0x7f569c0ccd40 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f5699ffb700 1 -- 172.21.15.89:0/1738108663 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257540591 0 0) 0x7f569c070e90 con 0x7f569c0cc7f0 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f569b7fe700 1 --2- 172.21.15.89:0/1738108663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c0ca2d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f569c070380 tx=0x7f569400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d421f874b41e085 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f5699ffb700 1 -- 172.21.15.89:0/1738108663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569400e070 con 0x7f569c0ccd40 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.165+0000 7f5699ffb700 1 -- 172.21.15.89:0/1738108663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569400bbd0 con 0x7f569c0ccd40 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f5699ffb700 1 -- 172.21.15.89:0/1738108663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694012430 con 0x7f569c0ccd40 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 -- 172.21.15.89:0/1738108663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 msgr2=0x7f569c0ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1738108663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c0ca2d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f569c070380 tx=0x7f569400b330 comp rx=0 tx=0).stop 2023-07-06T02:23:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 -- 172.21.15.89:0/1738108663 shutdown_connections 2023-07-06T02:23:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1738108663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c0ca2d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1738108663 >> v2:172.21.15.172:3300/0 conn(0x7f569c070580 0x7f569c070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 -- 172.21.15.89:0/1738108663 >> 172.21.15.89:0/1738108663 conn(0x7f569c0ce870 msgr2=0x7f569c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 -- 172.21.15.89:0/1738108663 shutdown_connections 2023-07-06T02:23:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.166+0000 7f56a17ed700 1 -- 172.21.15.89:0/1738108663 wait complete. 2023-07-06T02:23:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.167+0000 7f56a17ed700 1 Processor -- start 2023-07-06T02:23:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.167+0000 7f56a17ed700 1 -- start start 2023-07-06T02:23:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f56a17ed700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f569c070580 0x7f569c1f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f56a17ed700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f569c0cc7f0 0x7f569c149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f56a17ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c15cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f56a17ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0c6fc0 con 0x7f569c0ccd40 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f56a17ed700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0b3ef0 con 0x7f569c0cc7f0 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f56a17ed700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1f4510 con 0x7f569c070580 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c15cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c15cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34314/0 (socket says 172.21.15.89:34314) 2023-07-06T02:23:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f569b7fe700 1 -- 172.21.15.89:0/1584024564 learned_addr learned my addr 172.21.15.89:0/1584024564 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f569b7fe700 1 -- 172.21.15.89:0/1584024564 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f569c070580 msgr2=0x7f569c1f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f569b7fe700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f569c070580 0x7f569c1f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.168+0000 7f569b7fe700 1 -- 172.21.15.89:0/1584024564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f569c0cc7f0 msgr2=0x7f569c149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f569b7fe700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f569c0cc7f0 0x7f569c149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f569b7fe700 1 -- 172.21.15.89:0/1584024564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5694009a70 con 0x7f569c0ccd40 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f569b7fe700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c15cf00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5694014040 tx=0x7f5694004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f56837fe700 1 -- 172.21.15.89:0/1584024564 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694012430 con 0x7f569c0ccd40 2023-07-06T02:23:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f56837fe700 1 -- 172.21.15.89:0/1584024564 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5694020070 con 0x7f569c0ccd40 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f56837fe700 1 -- 172.21.15.89:0/1584024564 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694004280 con 0x7f569c0ccd40 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c14a0d0 con 0x7f569c0ccd40 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.169+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c0b4470 con 0x7f569c0ccd40 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.170+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c0b4690 con 0x7f569c0ccd40 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.170+0000 7f56837fe700 1 -- 172.21.15.89:0/1584024564 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56940043e0 con 0x7f569c0ccd40 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.171+0000 7f56837fe700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f568806ace0 0x7f568806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.171+0000 7f56837fe700 1 -- 172.21.15.89:0/1584024564 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5694022070 con 0x7f569c0ccd40 2023-07-06T02:23:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.171+0000 7f569affd700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f568806ace0 0x7f568806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.171+0000 7f569affd700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f568806ace0 0x7f568806d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5690009710 tx=0x7f5690006c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.175+0000 7f56837fe700 1 -- 172.21.15.89:0/1584024564 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5694056470 con 0x7f569c0ccd40 2023-07-06T02:23:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.363+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f569c066fb0 con 0x7f569c0ccd40 2023-07-06T02:23:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.364+0000 7f56837fe700 1 -- 172.21.15.89:0/1584024564 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f5694059a90 con 0x7f569c0ccd40 2023-07-06T02:23:56.367 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:56.367 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:23:41.389931+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f568806ace0 msgr2=0x7f568806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f568806ace0 0x7f568806d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5690009710 tx=0x7f5690006c80 comp rx=0 tx=0).stop 2023-07-06T02:23:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 msgr2=0x7f569c15cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c15cf00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5694014040 tx=0x7f5694004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 shutdown_connections 2023-07-06T02:23:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f568806ace0 0x7f568806d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f569c0ccd40 0x7f569c15cf00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f569c070580 0x7f569c1f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 --2- 172.21.15.89:0/1584024564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f569c0cc7f0 0x7f569c149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 >> 172.21.15.89:0/1584024564 conn(0x7f569c0ce870 msgr2=0x7f569c154200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 shutdown_connections 2023-07-06T02:23:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.366+0000 7f56a17ed700 1 -- 172.21.15.89:0/1584024564 wait complete. 2023-07-06T02:23:56.381 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T02:23:56.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:56.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.501+0000 7f37862f3700 1 Processor -- start 2023-07-06T02:23:56.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.501+0000 7f37862f3700 1 -- start start 2023-07-06T02:23:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.502+0000 7f37862f3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f37800705a0 0x7f37800ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.502+0000 7f37862f3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f378006dc10 0x7f3780157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.502+0000 7f37862f3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378006e110 con 0x7f37800cd180 2023-07-06T02:23:56.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.502+0000 7f37862f3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378006e3f0 con 0x7f37800705a0 2023-07-06T02:23:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.502+0000 7f37862f3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378006e6d0 con 0x7f378006dc10 2023-07-06T02:23:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f37800705a0 0x7f37800ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f3784890700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f378006dc10 0x7f3780157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f37800705a0 0x7f37800ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:33076/0 (socket says 172.21.15.89:33076) 2023-07-06T02:23:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 -- 172.21.15.89:0/2035632972 learned_addr learned my addr 172.21.15.89:0/2035632972 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377f7fe700 1 --1- 172.21.15.89:0/2035632972 >> v1:172.21.15.172:6789/0 conn(0x7f37800cd180 0x7f378015a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.89:34978/0 (socket says 172.21.15.89:34978) 2023-07-06T02:23:56.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 -- 172.21.15.89:0/2035632972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f378006dc10 msgr2=0x7f3780157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 --2- 172.21.15.89:0/2035632972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f378006dc10 0x7f3780157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 -- 172.21.15.89:0/2035632972 >> v1:172.21.15.172:6789/0 conn(0x7f37800cd180 legacy=0x7f378015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 -- 172.21.15.89:0/2035632972 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378006e9b0 con 0x7f37800705a0 2023-07-06T02:23:56.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.503+0000 7f377ffff700 1 --2- 172.21.15.89:0/2035632972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37800705a0 0x7f37800ccc40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f37680098f0 tx=0x7f3768009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1949ce8e9408fc48 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f377effd700 1 -- 172.21.15.89:0/2035632972 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768011830 con 0x7f37800705a0 2023-07-06T02:23:56.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f377effd700 1 -- 172.21.15.89:0/2035632972 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3768011990 con 0x7f37800705a0 2023-07-06T02:23:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f377effd700 1 -- 172.21.15.89:0/2035632972 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768011d20 con 0x7f37800705a0 2023-07-06T02:23:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 -- 172.21.15.89:0/2035632972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37800705a0 msgr2=0x7f37800ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 --2- 172.21.15.89:0/2035632972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37800705a0 0x7f37800ccc40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f37680098f0 tx=0x7f3768009c00 comp rx=0 tx=0).stop 2023-07-06T02:23:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 -- 172.21.15.89:0/2035632972 shutdown_connections 2023-07-06T02:23:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 --2- 172.21.15.89:0/2035632972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f378006dc10 0x7f3780157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 --2- 172.21.15.89:0/2035632972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37800705a0 0x7f37800ccc40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 -- 172.21.15.89:0/2035632972 >> 172.21.15.89:0/2035632972 conn(0x7f37800d0b00 msgr2=0x7f37800d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 -- 172.21.15.89:0/2035632972 shutdown_connections 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.504+0000 7f37862f3700 1 -- 172.21.15.89:0/2035632972 wait complete. 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.505+0000 7f37862f3700 1 Processor -- start 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f37862f3700 1 -- start start 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f37862f3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f378006dc10 0x7f37800c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f37862f3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37800705a0 0x7f37800c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f37862f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37800cd180 0x7f378006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f37862f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378006da40 con 0x7f37800cd180 2023-07-06T02:23:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f37862f3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37800c4c10 con 0x7f378006dc10 2023-07-06T02:23:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f37862f3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37800c4ef0 con 0x7f37800705a0 2023-07-06T02:23:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.506+0000 7f3784890700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37800cd180 0x7f378006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37800cd180 0x7f378006d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34328/0 (socket says 172.21.15.89:34328) 2023-07-06T02:23:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 -- 172.21.15.89:0/788630102 learned_addr learned my addr 172.21.15.89:0/788630102 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f377f7fe700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37800705a0 0x7f37800c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f377ffff700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f378006dc10 0x7f37800c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 -- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37800705a0 msgr2=0x7f37800c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37800705a0 0x7f37800c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 -- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f378006dc10 msgr2=0x7f37800c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f378006dc10 0x7f37800c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 -- 172.21.15.89:0/788630102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3768009550 con 0x7f37800cd180 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f377ffff700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f378006dc10 0x7f37800c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f3784890700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37800cd180 0x7f378006d500 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f377800efc0 tx=0x7f377800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f377cff9700 1 -- 172.21.15.89:0/788630102 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800cd40 con 0x7f37800cd180 2023-07-06T02:23:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f377cff9700 1 -- 172.21.15.89:0/788630102 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377800f040 con 0x7f37800cd180 2023-07-06T02:23:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37800c51d0 con 0x7f37800cd180 2023-07-06T02:23:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f377cff9700 1 -- 172.21.15.89:0/788630102 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37780146c0 con 0x7f37800cd180 2023-07-06T02:23:56.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.507+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37800c5790 con 0x7f37800cd180 2023-07-06T02:23:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.508+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3780074270 con 0x7f37800cd180 2023-07-06T02:23:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.509+0000 7f377cff9700 1 -- 172.21.15.89:0/788630102 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3778014820 con 0x7f37800cd180 2023-07-06T02:23:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.509+0000 7f377cff9700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f376006ace0 0x7f376006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.509+0000 7f377cff9700 1 -- 172.21.15.89:0/788630102 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f377808a4b0 con 0x7f37800cd180 2023-07-06T02:23:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.509+0000 7f377ffff700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f376006ace0 0x7f376006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.509+0000 7f377ffff700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f376006ace0 0x7f376006d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f37680098f0 tx=0x7f3768008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.513+0000 7f377cff9700 1 -- 172.21.15.89:0/788630102 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3778058c40 con 0x7f37800cd180 2023-07-06T02:23:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.743+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37801596b0 con 0x7f37800cd180 2023-07-06T02:23:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.744+0000 7f377cff9700 1 -- 172.21.15.89:0/788630102 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f377800cea0 con 0x7f37800cd180 2023-07-06T02:23:56.747 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:56.747 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f376006ace0 msgr2=0x7f376006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f376006ace0 0x7f376006d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f37680098f0 tx=0x7f3768008030 comp rx=0 tx=0).stop 2023-07-06T02:23:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37800cd180 msgr2=0x7f378006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37800cd180 0x7f378006d500 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f377800efc0 tx=0x7f377800c3a0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 shutdown_connections 2023-07-06T02:23:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f376006ace0 0x7f376006d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37800cd180 0x7f378006d500 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f378006dc10 0x7f37800c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 --2- 172.21.15.89:0/788630102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37800705a0 0x7f37800c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 >> 172.21.15.89:0/788630102 conn(0x7f37800d0b00 msgr2=0x7f378015c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 shutdown_connections 2023-07-06T02:23:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.746+0000 7f37862f3700 1 -- 172.21.15.89:0/788630102 wait complete. 2023-07-06T02:23:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:56.761 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:23:56.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.877+0000 7fa391eee700 1 Processor -- start 2023-07-06T02:23:56.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.877+0000 7fa391eee700 1 -- start start 2023-07-06T02:23:56.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa391eee700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa38c0705a0 0x7fa38c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa391eee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c06dc10 0x7fa38c157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa391eee700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38c06e110 con 0x7fa38c0cd180 2023-07-06T02:23:56.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa391eee700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38c06e3f0 con 0x7fa38c0705a0 2023-07-06T02:23:56.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa391eee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c06e6d0 con 0x7fa38c06dc10 2023-07-06T02:23:56.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa38b7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa38c0705a0 0x7fa38c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa38affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa38c0cd180 0x7fa38c15a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59234/0 (socket says 172.21.15.89:59234) 2023-07-06T02:23:56.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.878+0000 7fa38affd700 1 -- 172.21.15.89:0/1913332873 learned_addr learned my addr 172.21.15.89:0/1913332873 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:56.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38bfff700 1 --2- 172.21.15.89:0/1913332873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c06dc10 0x7fa38c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38b7fe700 1 -- 172.21.15.89:0/1913332873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c06dc10 msgr2=0x7fa38c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:56.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38b7fe700 1 --2- 172.21.15.89:0/1913332873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c06dc10 0x7fa38c157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38b7fe700 1 -- 172.21.15.89:0/1913332873 >> v1:172.21.15.89:6789/0 conn(0x7fa38c0cd180 legacy=0x7fa38c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38a7fc700 1 -- 172.21.15.89:0/1913332873 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850952608 0 0) 0x7fa38c06e110 con 0x7fa38c0cd180 2023-07-06T02:23:56.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38b7fe700 1 -- 172.21.15.89:0/1913332873 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c06e9b0 con 0x7fa38c0705a0 2023-07-06T02:23:56.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38b7fe700 1 --2- 172.21.15.89:0/1913332873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0705a0 0x7fa38c0ccc40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa374009840 tx=0x7fa374009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb449be89da0ce2b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38a7fc700 1 -- 172.21.15.89:0/1913332873 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374011830 con 0x7fa38c0705a0 2023-07-06T02:23:56.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38a7fc700 1 -- 172.21.15.89:0/1913332873 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374011990 con 0x7fa38c0705a0 2023-07-06T02:23:56.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa38a7fc700 1 -- 172.21.15.89:0/1913332873 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374011d20 con 0x7fa38c0705a0 2023-07-06T02:23:56.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa391eee700 1 -- 172.21.15.89:0/1913332873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0705a0 msgr2=0x7fa38c0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.879+0000 7fa391eee700 1 --2- 172.21.15.89:0/1913332873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0705a0 0x7fa38c0ccc40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa374009840 tx=0x7fa374009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:56.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.880+0000 7fa391eee700 1 -- 172.21.15.89:0/1913332873 shutdown_connections 2023-07-06T02:23:56.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.880+0000 7fa391eee700 1 --2- 172.21.15.89:0/1913332873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c06dc10 0x7fa38c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.880+0000 7fa391eee700 1 --2- 172.21.15.89:0/1913332873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0705a0 0x7fa38c0ccc40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.880+0000 7fa391eee700 1 -- 172.21.15.89:0/1913332873 >> 172.21.15.89:0/1913332873 conn(0x7fa38c0d0b00 msgr2=0x7fa38c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:56.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.880+0000 7fa391eee700 1 -- 172.21.15.89:0/1913332873 shutdown_connections 2023-07-06T02:23:56.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.880+0000 7fa391eee700 1 -- 172.21.15.89:0/1913332873 wait complete. 2023-07-06T02:23:56.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.881+0000 7fa391eee700 1 Processor -- start 2023-07-06T02:23:56.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.881+0000 7fa391eee700 1 -- start start 2023-07-06T02:23:56.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38c06dc10 0x7fa38c1ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c0705a0 0x7fa38c0c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0cd180 0x7fa38c0c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c14a5d0 con 0x7fa38c0cd180 2023-07-06T02:23:56.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c0c7d50 con 0x7fa38c06dc10 2023-07-06T02:23:56.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c0c8030 con 0x7fa38c0705a0 2023-07-06T02:23:56.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0cd180 0x7fa38c0c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0cd180 0x7fa38c0c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34340/0 (socket says 172.21.15.89:34340) 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 -- 172.21.15.89:0/936048747 learned_addr learned my addr 172.21.15.89:0/936048747 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38b7fe700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38c06dc10 0x7fa38c1ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38affd700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c0705a0 0x7fa38c0c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 -- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c0705a0 msgr2=0x7fa38c0c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c0705a0 0x7fa38c0c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 -- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38c06dc10 msgr2=0x7fa38c1ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:56.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38c06dc10 0x7fa38c1ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 -- 172.21.15.89:0/936048747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740094a0 con 0x7fa38c0cd180 2023-07-06T02:23:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa38bfff700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0cd180 0x7fa38c0c6a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa384007e40 tx=0x7fa38400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa383fff700 1 -- 172.21.15.89:0/936048747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38400ede0 con 0x7fa38c0cd180 2023-07-06T02:23:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa383fff700 1 -- 172.21.15.89:0/936048747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384014420 con 0x7fa38c0cd180 2023-07-06T02:23:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38c0c8310 con 0x7fa38c0cd180 2023-07-06T02:23:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa383fff700 1 -- 172.21.15.89:0/936048747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3840147d0 con 0x7fa38c0cd180 2023-07-06T02:23:56.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.882+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c0c4630 con 0x7fa38c0cd180 2023-07-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.883+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38c0c8530 con 0x7fa38c0cd180 2023-07-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.884+0000 7fa383fff700 1 -- 172.21.15.89:0/936048747 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa384014930 con 0x7fa38c0cd180 2023-07-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.884+0000 7fa383fff700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06ace0 0x7fa36c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.884+0000 7fa383fff700 1 -- 172.21.15.89:0/936048747 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa384089f50 con 0x7fa38c0cd180 2023-07-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.884+0000 7fa38b7fe700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06ace0 0x7fa36c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:56.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.885+0000 7fa38b7fe700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06ace0 0x7fa36c06d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa3740090b0 tx=0x7fa374008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:56.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:56.888+0000 7fa383fff700 1 -- 172.21.15.89:0/936048747 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa384058810 con 0x7fa38c0cd180 2023-07-06T02:23:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.116+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa38c074270 con 0x7fa38c0cd180 2023-07-06T02:23:57.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.117+0000 7fa383fff700 1 -- 172.21.15.89:0/936048747 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fa3840583a0 con 0x7fa38c0cd180 2023-07-06T02:23:57.120 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:57.120 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/4028435760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4028435760},{"type":"v1","addr":"172.21.15.172:6834","nonce":4028435760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/679255822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":679255822},{"type":"v1","addr":"172.21.15.172:6835","nonce":679255822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2843534431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2843534431},{"type":"v1","addr":"172.21.15.89:6835","nonce":2843534431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:22:52.797894+0000","modified":"2023-07-06T02:23:11.177784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/1524050092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1524050092},{"type":"v1","addr":"172.21.15.89:6837","nonce":1524050092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T02:23:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06ace0 msgr2=0x7fa36c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06ace0 0x7fa36c06d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa3740090b0 tx=0x7fa374008030 comp rx=0 tx=0).stop 2023-07-06T02:23:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0cd180 msgr2=0x7fa38c0c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0cd180 0x7fa38c0c6a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa384007e40 tx=0x7fa38400e3c0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 shutdown_connections 2023-07-06T02:23:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa36c06ace0 0x7fa36c06d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c0cd180 0x7fa38c0c6a10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa38c06dc10 0x7fa38c1ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 --2- 172.21.15.89:0/936048747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38c0705a0 0x7fa38c0c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 >> 172.21.15.89:0/936048747 conn(0x7fa38c0d0b00 msgr2=0x7fa38c0b61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 shutdown_connections 2023-07-06T02:23:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.119+0000 7fa391eee700 1 -- 172.21.15.89:0/936048747 wait complete. 2023-07-06T02:23:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-07-06T02:23:57.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:23:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.251+0000 7f03d5c65700 1 Processor -- start 2023-07-06T02:23:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.251+0000 7f03d5c65700 1 -- start start 2023-07-06T02:23:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03d5c65700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03d00b1490 0x7f03d00caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03d5c65700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d006b220 0x7f03d015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03d5c65700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d006a990 con 0x7f03d00cc000 2023-07-06T02:23:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03d5c65700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d006ac70 con 0x7f03d00b1490 2023-07-06T02:23:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03d5c65700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d006d170 con 0x7f03d006b220 2023-07-06T02:23:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03d00b1490 0x7f03d00caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cf7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f03d00b1490 0x7f03d00caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34344/0 (socket says 172.21.15.89:34344) 2023-07-06T02:23:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cf7fe700 1 -- 172.21.15.89:0/2828192830 learned_addr learned my addr 172.21.15.89:0/2828192830 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cffff700 1 --2- 172.21.15.89:0/2828192830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d006b220 0x7f03d015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cf7fe700 1 -- 172.21.15.89:0/2828192830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d006b220 msgr2=0x7f03d015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cf7fe700 1 --2- 172.21.15.89:0/2828192830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d006b220 0x7f03d015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cf7fe700 1 -- 172.21.15.89:0/2828192830 >> v1:172.21.15.172:6789/0 conn(0x7f03d00cc000 legacy=0x7f03d00b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.252+0000 7f03cf7fe700 1 -- 172.21.15.89:0/2828192830 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d006ce60 con 0x7f03d00b1490 2023-07-06T02:23:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03cf7fe700 1 --2- 172.21.15.89:0/2828192830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d00b1490 0x7f03d00caeb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009e50 tx=0x7f03c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3ba10d60aa3ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03ce7fc700 1 -- 172.21.15.89:0/2828192830 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c000e070 con 0x7f03d00b1490 2023-07-06T02:23:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03ce7fc700 1 -- 172.21.15.89:0/2828192830 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c000b9c0 con 0x7f03d00b1490 2023-07-06T02:23:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03ce7fc700 1 -- 172.21.15.89:0/2828192830 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c000bd70 con 0x7f03d00b1490 2023-07-06T02:23:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 -- 172.21.15.89:0/2828192830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d00b1490 msgr2=0x7f03d00caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 --2- 172.21.15.89:0/2828192830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d00b1490 0x7f03d00caeb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009e50 tx=0x7f03c000b330 comp rx=0 tx=0).stop 2023-07-06T02:23:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 -- 172.21.15.89:0/2828192830 shutdown_connections 2023-07-06T02:23:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 --2- 172.21.15.89:0/2828192830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d006b220 0x7f03d015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 --2- 172.21.15.89:0/2828192830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d00b1490 0x7f03d00caeb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 -- 172.21.15.89:0/2828192830 >> 172.21.15.89:0/2828192830 conn(0x7f03d00cc490 msgr2=0x7f03d00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 -- 172.21.15.89:0/2828192830 shutdown_connections 2023-07-06T02:23:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.253+0000 7f03d5c65700 1 -- 172.21.15.89:0/2828192830 wait complete. 2023-07-06T02:23:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.255+0000 7f03d5c65700 1 Processor -- start 2023-07-06T02:23:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.255+0000 7f03d5c65700 1 -- start start 2023-07-06T02:23:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03d5c65700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d006b220 0x7f03d00c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03d5c65700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d00b1490 0x7f03d00a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03d5c65700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d00cc000 0x7f03d014c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03d5c65700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015c750 con 0x7f03d006b220 2023-07-06T02:23:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03d5c65700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015bb40 con 0x7f03d00b1490 2023-07-06T02:23:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03d5c65700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015be20 con 0x7f03d00cc000 2023-07-06T02:23:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d006b220 0x7f03d00c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d006b220 0x7f03d00c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34348/0 (socket says 172.21.15.89:34348) 2023-07-06T02:23:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 -- 172.21.15.89:0/542444890 learned_addr learned my addr 172.21.15.89:0/542444890 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cffff700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d00cc000 0x7f03d014c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03ceffd700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d00b1490 0x7f03d00a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 -- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d00cc000 msgr2=0x7f03d014c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d00cc000 0x7f03d014c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 -- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d00b1490 msgr2=0x7f03d00a74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d00b1490 0x7f03d00a74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.256+0000 7f03cf7fe700 1 -- 172.21.15.89:0/542444890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0009b30 con 0x7f03d006b220 2023-07-06T02:23:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.257+0000 7f03cf7fe700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d006b220 0x7f03d00c37b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f03c0000f80 tx=0x7f03c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.257+0000 7f03c7fff700 1 -- 172.21.15.89:0/542444890 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00099b0 con 0x7f03d006b220 2023-07-06T02:23:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.257+0000 7f03c7fff700 1 -- 172.21.15.89:0/542444890 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c00092e0 con 0x7f03d006b220 2023-07-06T02:23:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.257+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d015c100 con 0x7f03d006b220 2023-07-06T02:23:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.257+0000 7f03c7fff700 1 -- 172.21.15.89:0/542444890 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c001c660 con 0x7f03d006b220 2023-07-06T02:23:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.257+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d0159520 con 0x7f03d006b220 2023-07-06T02:23:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.258+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d0074270 con 0x7f03d006b220 2023-07-06T02:23:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.258+0000 7f03c7fff700 1 -- 172.21.15.89:0/542444890 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03c001c7c0 con 0x7f03d006b220 2023-07-06T02:23:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.259+0000 7f03c7fff700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f03b006ace0 0x7f03b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.259+0000 7f03c7fff700 1 -- 172.21.15.89:0/542444890 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f03c008ae30 con 0x7f03d006b220 2023-07-06T02:23:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.259+0000 7f03ceffd700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f03b006ace0 0x7f03b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.259+0000 7f03ceffd700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f03b006ace0 0x7f03b006d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03d00c5140 tx=0x7f03b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:57.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.263+0000 7f03c7fff700 1 -- 172.21.15.89:0/542444890 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f03c0055bb0 con 0x7f03d006b220 2023-07-06T02:23:57.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.454+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03d00ca280 con 0x7f03d006b220 2023-07-06T02:23:57.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.458+0000 7f03c7fff700 1 -- 172.21.15.89:0/542444890 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f03c00591d0 con 0x7f03d006b220 2023-07-06T02:23:57.461 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:23:57.461 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:23:41.389931+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:22:50.488557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T02:22:51.770162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-06T02:22:55.444256+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:23:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f03b006ace0 msgr2=0x7f03b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f03b006ace0 0x7f03b006d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03d00c5140 tx=0x7f03b8009040 comp rx=0 tx=0).stop 2023-07-06T02:23:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d006b220 msgr2=0x7f03d00c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d006b220 0x7f03d00c37b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f03c0000f80 tx=0x7f03c0004ad0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 shutdown_connections 2023-07-06T02:23:57.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03d00cc000 0x7f03d014c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f03b006ace0 0x7f03b006d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03d006b220 0x7f03d00c37b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 --2- 172.21.15.89:0/542444890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d00b1490 0x7f03d00a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 >> 172.21.15.89:0/542444890 conn(0x7f03d00cc490 msgr2=0x7f03d0159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 shutdown_connections 2023-07-06T02:23:57.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.460+0000 7f03d5c65700 1 -- 172.21.15.89:0/542444890 wait complete. 2023-07-06T02:23:57.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T02:23:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.595+0000 7efe52a66700 1 Processor -- start 2023-07-06T02:23:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.595+0000 7efe52a66700 1 -- start start 2023-07-06T02:23:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe52a66700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efe4c0b1490 0x7efe4c0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe52a66700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c0cab50 con 0x7efe4c15c690 2023-07-06T02:23:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe52a66700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c15db70 con 0x7efe4c06bfa0 2023-07-06T02:23:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe52a66700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4c15de50 con 0x7efe4c0b1490 2023-07-06T02:23:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe50802700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efe4c15c690 0x7efe4c0cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59258/0 (socket says 172.21.15.89:59258) 2023-07-06T02:23:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe50802700 1 -- 172.21.15.89:0/1217167764 learned_addr learned my addr 172.21.15.89:0/1217167764 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe4bfff700 1 --2- 172.21.15.89:0/1217167764 >> v2:172.21.15.172:3300/0 conn(0x7efe4c0b1490 0x7efe4c0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe4b7fe700 1 -- 172.21.15.89:0/1217167764 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971069755 0 0) 0x7efe4c0cab50 con 0x7efe4c15c690 2023-07-06T02:23:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.596+0000 7efe4b7fe700 1 -- 172.21.15.89:0/1217167764 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe3c0035b0 con 0x7efe4c15c690 2023-07-06T02:23:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4b7fe700 1 -- 172.21.15.89:0/1217167764 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2861814686 0 0) 0x7efe3c0035b0 con 0x7efe4c15c690 2023-07-06T02:23:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4bfff700 1 -- 172.21.15.89:0/1217167764 >> v1:172.21.15.172:6789/0 conn(0x7efe4c06bfa0 legacy=0x7efe4c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4bfff700 1 -- 172.21.15.89:0/1217167764 >> v1:172.21.15.89:6789/0 conn(0x7efe4c15c690 legacy=0x7efe4c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4bfff700 1 -- 172.21.15.89:0/1217167764 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0cad80 con 0x7efe4c0b1490 2023-07-06T02:23:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4bfff700 1 --2- 172.21.15.89:0/1217167764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0b1910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe38009840 tx=0x7efe38009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b71fb10226d169c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4b7fe700 1 -- 172.21.15.89:0/1217167764 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40001670 con 0x7efe4c0b1490 2023-07-06T02:23:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4b7fe700 1 -- 172.21.15.89:0/1217167764 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38004030 con 0x7efe4c0b1490 2023-07-06T02:23:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.597+0000 7efe4b7fe700 1 -- 172.21.15.89:0/1217167764 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe380043c0 con 0x7efe4c0b1490 2023-07-06T02:23:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.598+0000 7efe52a66700 1 -- 172.21.15.89:0/1217167764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 msgr2=0x7efe4c0b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.598+0000 7efe52a66700 1 --2- 172.21.15.89:0/1217167764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0b1910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe38009840 tx=0x7efe38009b50 comp rx=0 tx=0).stop 2023-07-06T02:23:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.598+0000 7efe52a66700 1 -- 172.21.15.89:0/1217167764 shutdown_connections 2023-07-06T02:23:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.598+0000 7efe52a66700 1 --2- 172.21.15.89:0/1217167764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0b1910 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.598+0000 7efe52a66700 1 -- 172.21.15.89:0/1217167764 >> 172.21.15.89:0/1217167764 conn(0x7efe4c0cc490 msgr2=0x7efe4c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.598+0000 7efe52a66700 1 -- 172.21.15.89:0/1217167764 shutdown_connections 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.598+0000 7efe52a66700 1 -- 172.21.15.89:0/1217167764 wait complete. 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.599+0000 7efe52a66700 1 Processor -- start 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.599+0000 7efe52a66700 1 -- start start 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe52a66700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4c06bfa0 0x7efe4c0afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe52a66700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe52a66700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4c15c690 0x7efe4c0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe52a66700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c1f0250 con 0x7efe4c06bfa0 2023-07-06T02:23:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe52a66700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0b0320 con 0x7efe4c0b1490 2023-07-06T02:23:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe52a66700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0c6720 con 0x7efe4c15c690 2023-07-06T02:23:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4c06bfa0 0x7efe4c0afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4c06bfa0 0x7efe4c0afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34362/0 (socket says 172.21.15.89:34362) 2023-07-06T02:23:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 -- 172.21.15.89:0/1960921721 learned_addr learned my addr 172.21.15.89:0/1960921721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe4bfff700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe51003700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4c15c690 0x7efe4c0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 -- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4c15c690 msgr2=0x7efe4c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4c15c690 0x7efe4c0cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 -- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 msgr2=0x7efe4c0a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 -- 172.21.15.89:0/1960921721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380094a0 con 0x7efe4c06bfa0 2023-07-06T02:23:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe4bfff700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:57.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe50802700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4c06bfa0 0x7efe4c0afde0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c730 tx=0x7efe4000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:57.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe497fa700 1 -- 172.21.15.89:0/1960921721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40004c80 con 0x7efe4c06bfa0 2023-07-06T02:23:57.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe497fa700 1 -- 172.21.15.89:0/1960921721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe40004e00 con 0x7efe4c06bfa0 2023-07-06T02:23:57.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c0c5ba0 con 0x7efe4c06bfa0 2023-07-06T02:23:57.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.600+0000 7efe497fa700 1 -- 172.21.15.89:0/1960921721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40007310 con 0x7efe4c06bfa0 2023-07-06T02:23:57.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.601+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0c6440 con 0x7efe4c06bfa0 2023-07-06T02:23:57.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.602+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c066fb0 con 0x7efe4c06bfa0 2023-07-06T02:23:57.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.602+0000 7efe497fa700 1 -- 172.21.15.89:0/1960921721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe40007470 con 0x7efe4c06bfa0 2023-07-06T02:23:57.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.602+0000 7efe497fa700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efe3006ace0 0x7efe3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.602+0000 7efe497fa700 1 -- 172.21.15.89:0/1960921721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efe40088c20 con 0x7efe4c06bfa0 2023-07-06T02:23:57.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.602+0000 7efe4bfff700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efe3006ace0 0x7efe3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.603+0000 7efe4bfff700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efe3006ace0 0x7efe3006d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0c8f90 tx=0x7efe38008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:57.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.606+0000 7efe497fa700 1 -- 172.21.15.89:0/1960921721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efe40053a30 con 0x7efe4c06bfa0 2023-07-06T02:23:57.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.836+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efe4c074270 con 0x7efe4c06bfa0 2023-07-06T02:23:57.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-07-06T02:23:57.837+0000 7f743e649700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T02:23:57.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.845+0000 7efe497fa700 1 -- 172.21.15.89:0/1960921721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7efe40057050 con 0x7efe4c06bfa0 2023-07-06T02:23:57.848 INFO:tasks.ceph.mds.c.smithi089.stderr: -5736> 2023-07-06T02:22:58.457+0000 7fc1fcf4e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T02:23:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efe3006ace0 msgr2=0x7efe3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efe3006ace0 0x7efe3006d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0c8f90 tx=0x7efe38008000 comp rx=0 tx=0).stop 2023-07-06T02:23:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4c06bfa0 msgr2=0x7efe4c0afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4c06bfa0 0x7efe4c0afde0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c730 tx=0x7efe4000caf0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 shutdown_connections 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4c15c690 0x7efe4c0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7efe3006ace0 0x7efe3006d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe4c06bfa0 0x7efe4c0afde0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 --2- 172.21.15.89:0/1960921721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4c0b1490 0x7efe4c0a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 >> 172.21.15.89:0/1960921721 conn(0x7efe4c0cc490 msgr2=0x7efe4c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 shutdown_connections 2023-07-06T02:23:57.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.847+0000 7efe52a66700 1 -- 172.21.15.89:0/1960921721 wait complete. 2023-07-06T02:23:57.852 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T02:23:57.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T02:23:57.878 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-07-06T02:23:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.982+0000 7f621375a700 1 Processor -- start 2023-07-06T02:23:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.982+0000 7f621375a700 1 -- start start 2023-07-06T02:23:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f621375a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 0x7f620c157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f621375a700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c0cd350 con 0x7f620c0705a0 2023-07-06T02:23:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f621375a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c06e110 con 0x7f620c06dc10 2023-07-06T02:23:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f621375a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c06e3f0 con 0x7f620c070970 2023-07-06T02:23:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f62114f6700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f620c0705a0 0x7f620c0ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59284/0 (socket says 172.21.15.89:59284) 2023-07-06T02:23:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f62114f6700 1 -- 172.21.15.89:0/1130910033 learned_addr learned my addr 172.21.15.89:0/1130910033 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f6211cf7700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 0x7f620c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.983+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860012434 0 0) 0x7f620c0cd350 con 0x7f620c0705a0 2023-07-06T02:23:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f80035b0 con 0x7f620c0705a0 2023-07-06T02:23:57.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493683161 0 0) 0x7f61f80035b0 con 0x7f620c0705a0 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 msgr2=0x7f620c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 0x7f620c157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 >> v1:172.21.15.172:6789/0 conn(0x7f620c06dc10 legacy=0x7f620c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c06e6d0 con 0x7f620c0705a0 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f61fc001670 con 0x7f620c0705a0 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 >> v1:172.21.15.89:6789/0 conn(0x7f620c0705a0 legacy=0x7f620c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f8003a30 0x7f61f8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f61f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 0x7f61f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c06e3f0 con 0x7f61f8004850 2023-07-06T02:23:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f800bac0 con 0x7f61f8003a30 2023-07-06T02:23:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f800bc50 con 0x7f620c070970 2023-07-06T02:23:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6210cf5700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f61f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f62114f6700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f8003a30 0x7f61f8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6211cf7700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 0x7f61f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6210cf5700 1 -- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 msgr2=0x7f61f8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6210cf5700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c070970 0x7f61f8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6210cf5700 1 -- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f8003a30 msgr2=0x7f61f8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6210cf5700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f8003a30 0x7f61f8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6210cf5700 1 -- 172.21.15.89:0/1130910033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80035b0 con 0x7f61f8004850 2023-07-06T02:23:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6210cf5700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f61f8004cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f62080017c0 tx=0x7f61f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e1baeb33e6e5c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:57.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.984+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62080052d0 con 0x7f61f8004850 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6208005430 con 0x7f61f8004850 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f6203fff700 1 -- 172.21.15.89:0/1130910033 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62080038c0 con 0x7f61f8004850 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f621375a700 1 -- 172.21.15.89:0/1130910033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 msgr2=0x7f61f8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f621375a700 1 --2- 172.21.15.89:0/1130910033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f61f8004cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f62080017c0 tx=0x7f61f8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f6211cf7700 1 -- 172.21.15.89:0/1130910033 reap_dead start 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f621375a700 1 -- 172.21.15.89:0/1130910033 shutdown_connections 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f621375a700 1 -- 172.21.15.89:0/1130910033 >> 172.21.15.89:0/1130910033 conn(0x7f620c0d0b00 msgr2=0x7f620c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:57.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f621375a700 1 -- 172.21.15.89:0/1130910033 shutdown_connections 2023-07-06T02:23:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.985+0000 7f621375a700 1 -- 172.21.15.89:0/1130910033 wait complete. 2023-07-06T02:23:57.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.986+0000 7f621375a700 1 Processor -- start 2023-07-06T02:23:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.986+0000 7f621375a700 1 -- start start 2023-07-06T02:23:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f621375a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f620c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f621375a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1effa0 0x7f620c1f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f621375a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c158210 0x7f620c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f621375a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c158bf0 con 0x7f61f8004850 2023-07-06T02:23:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f621375a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0caa40 con 0x7f620c1effa0 2023-07-06T02:23:57.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f621375a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0c9b50 con 0x7f620c158210 2023-07-06T02:23:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f620c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f620c157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34394/0 (socket says 172.21.15.89:34394) 2023-07-06T02:23:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 -- 172.21.15.89:0/2748964074 learned_addr learned my addr 172.21.15.89:0/2748964074 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 -- 172.21.15.89:0/2748964074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c158210 msgr2=0x7f620c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c158210 0x7f620c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 -- 172.21.15.89:0/2748964074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1effa0 msgr2=0x7f620c1f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1effa0 0x7f620c1f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:57.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.987+0000 7f62114f6700 1 -- 172.21.15.89:0/2748964074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0c9870 con 0x7f61f8004850 2023-07-06T02:23:57.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.988+0000 7f62114f6700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f620c157190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f61fc007290 tx=0x7f61fc0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.988+0000 7f6201ffb700 1 -- 172.21.15.89:0/2748964074 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00f5e0 con 0x7f61f8004850 2023-07-06T02:23:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.988+0000 7f6201ffb700 1 -- 172.21.15.89:0/2748964074 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc00f740 con 0x7f61f8004850 2023-07-06T02:23:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.988+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c0c46d0 con 0x7f61f8004850 2023-07-06T02:23:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.988+0000 7f6201ffb700 1 -- 172.21.15.89:0/2748964074 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00faf0 con 0x7f61f8004850 2023-07-06T02:23:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.988+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c0c4c90 con 0x7f61f8004850 2023-07-06T02:23:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.989+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c074270 con 0x7f61f8004850 2023-07-06T02:23:58.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.989+0000 7f6201ffb700 1 -- 172.21.15.89:0/2748964074 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61fc00fc50 con 0x7f61f8004850 2023-07-06T02:23:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.990+0000 7f6201ffb700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f61f006ace0 0x7f61f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.990+0000 7f6201ffb700 1 -- 172.21.15.89:0/2748964074 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f61fc0896b0 con 0x7f61f8004850 2023-07-06T02:23:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.990+0000 7f6210cf5700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f61f006ace0 0x7f61f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:58.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.990+0000 7f6210cf5700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f61f006ace0 0x7f61f006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f620c159220 tx=0x7f6208011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:58.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:57.994+0000 7f6201ffb700 1 -- 172.21.15.89:0/2748964074 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f61fc054470 con 0x7f61f8004850 2023-07-06T02:23:58.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.229+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f620c06f590 con 0x7f61f8004850 2023-07-06T02:23:58.343 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:23:58.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:23:58.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.850+0000 7f6201ffb700 1 -- 172.21.15.89:0/2748964074 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61fc057a90 con 0x7f61f8004850 2023-07-06T02:23:58.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.852+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f61f006ace0 msgr2=0x7f61f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:58.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.852+0000 7f621375a700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f61f006ace0 0x7f61f006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f620c159220 tx=0x7f6208011000 comp rx=0 tx=0).stop 2023-07-06T02:23:58.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.852+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 msgr2=0x7f620c157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:58.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.852+0000 7f621375a700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f620c157190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f61fc007290 tx=0x7f61fc0075a0 comp rx=0 tx=0).stop 2023-07-06T02:23:58.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 shutdown_connections 2023-07-06T02:23:58.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c158210 0x7f620c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:58.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f61f006ace0 0x7f61f006d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:58.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61f8004850 0x7f620c157190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:58.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 --2- 172.21.15.89:0/2748964074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1effa0 0x7f620c1f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:58.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 >> 172.21.15.89:0/2748964074 conn(0x7f620c0d0b00 msgr2=0x7f620c159960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:58.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 shutdown_connections 2023-07-06T02:23:58.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.853+0000 7f621375a700 1 -- 172.21.15.89:0/2748964074 wait complete. 2023-07-06T02:23:58.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T02:23:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.992+0000 7f0205220700 1 Processor -- start 2023-07-06T02:23:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.992+0000 7f0205220700 1 -- start start 2023-07-06T02:23:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f0205220700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f02000b2cd0 0x7f02000ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f0205220700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02000cf880 con 0x7f02000cb100 2023-07-06T02:23:58.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f0205220700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02000cb590 con 0x7f020015a420 2023-07-06T02:23:58.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f0205220700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02000cb870 con 0x7f02000b2cd0 2023-07-06T02:23:58.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f01fe59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f02000cb100 0x7f02000cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59308/0 (socket says 172.21.15.89:59308) 2023-07-06T02:23:58.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f01fe59c700 1 -- 172.21.15.89:0/3752435368 learned_addr learned my addr 172.21.15.89:0/3752435368 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:23:58.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f01ff59e700 1 --2- 172.21.15.89:0/3752435368 >> v2:172.21.15.172:3300/0 conn(0x7f02000b2cd0 0x7f02000ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:58.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785715518 0 0) 0x7f02000cf880 con 0x7f02000cb100 2023-07-06T02:23:58.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.993+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e80035b0 con 0x7f02000cb100 2023-07-06T02:23:59.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3883021520 0 0) 0x7f01e80035b0 con 0x7f02000cb100 2023-07-06T02:23:59.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 >> v2:172.21.15.172:3300/0 conn(0x7f02000b2cd0 msgr2=0x7f02000ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:59.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 --2- 172.21.15.89:0/3752435368 >> v2:172.21.15.172:3300/0 conn(0x7f02000b2cd0 0x7f02000ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:59.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 >> v1:172.21.15.172:6789/0 conn(0x7f020015a420 legacy=0x7f02000b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:59.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000cf230 con 0x7f02000cb100 2023-07-06T02:23:59.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f01f0001670 con 0x7f02000cb100 2023-07-06T02:23:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 >> v1:172.21.15.89:6789/0 conn(0x7f02000cb100 legacy=0x7f02000cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e8003a00 0x7f01e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e80043f0 0x7f01e80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01e80050b0 0x7f01e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000cb870 con 0x7f01e8003a00 2023-07-06T02:23:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e800c000 con 0x7f01e80050b0 2023-07-06T02:23:59.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e800c1c0 con 0x7f01e80043f0 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fed9d700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e8003a00 0x7f01e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01ff59e700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01e80050b0 0x7f01e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fe59c700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e80043f0 0x7f01e80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fed9d700 1 -- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e80043f0 msgr2=0x7f01e80048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fed9d700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e80043f0 0x7f01e80048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fed9d700 1 -- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01e80050b0 msgr2=0x7f01e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fed9d700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01e80050b0 0x7f01e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:59.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fed9d700 1 -- 172.21.15.89:0/3752435368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80035b0 con 0x7f01e8003a00 2023-07-06T02:23:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01ff59e700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01e80050b0 0x7f01e800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:23:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01ff59e700 1 -- 172.21.15.89:0/3752435368 reap_dead start 2023-07-06T02:23:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fed9d700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e8003a00 0x7f01e8003dd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f01f40018d0 tx=0x7f01e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90587208fc21ffe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40052d0 con 0x7f01e8003a00 2023-07-06T02:23:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f4005430 con 0x7f01e8003a00 2023-07-06T02:23:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.994+0000 7f01fdd9b700 1 -- 172.21.15.89:0/3752435368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40038c0 con 0x7f01e8003a00 2023-07-06T02:23:59.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.995+0000 7f0205220700 1 -- 172.21.15.89:0/3752435368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e8003a00 msgr2=0x7f01e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:59.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.995+0000 7f0205220700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e8003a00 0x7f01e8003dd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f01f40018d0 tx=0x7f01e8005880 comp rx=0 tx=0).stop 2023-07-06T02:23:59.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.995+0000 7f0205220700 1 -- 172.21.15.89:0/3752435368 shutdown_connections 2023-07-06T02:23:59.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.995+0000 7f0205220700 1 --2- 172.21.15.89:0/3752435368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e8003a00 0x7f01e8003dd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:59.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.995+0000 7f0205220700 1 -- 172.21.15.89:0/3752435368 >> 172.21.15.89:0/3752435368 conn(0x7f02000cd5c0 msgr2=0x7f02000cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:23:59.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.995+0000 7f0205220700 1 -- 172.21.15.89:0/3752435368 shutdown_connections 2023-07-06T02:23:59.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.995+0000 7f0205220700 1 -- 172.21.15.89:0/3752435368 wait complete. 2023-07-06T02:23:59.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.997+0000 7f0205220700 1 Processor -- start 2023-07-06T02:23:59.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.997+0000 7f0205220700 1 -- start start 2023-07-06T02:23:59.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f0205220700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e8003a00 0x7f02000c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:59.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f0205220700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020014bc70 0x7f020014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:59.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f0205220700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02000c8a40 0x7f02000c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:59.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f0205220700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000ca330 con 0x7f02000c8a40 2023-07-06T02:23:59.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f0205220700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020014c620 con 0x7f020014bc70 2023-07-06T02:23:59.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f0205220700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c97b0 con 0x7f01e8003a00 2023-07-06T02:23:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02000c8a40 0x7f02000c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02000c8a40 0x7f02000c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34420/0 (socket says 172.21.15.89:34420) 2023-07-06T02:23:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 -- 172.21.15.89:0/448470831 learned_addr learned my addr 172.21.15.89:0/448470831 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:23:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01fed9d700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e8003a00 0x7f02000c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01fe59c700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020014bc70 0x7f020014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 -- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e8003a00 msgr2=0x7f02000c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:59.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e8003a00 0x7f02000c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:59.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 -- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020014bc70 msgr2=0x7f020014c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:23:59.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020014bc70 0x7f020014c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:23:59.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01ff59e700 1 -- 172.21.15.89:0/448470831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000c9a90 con 0x7f02000c8a40 2023-07-06T02:23:59.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01fed9d700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e8003a00 0x7f02000c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:59.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.998+0000 7f01fe59c700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020014bc70 0x7f020014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:23:59.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.999+0000 7f01ff59e700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02000c8a40 0x7f02000c8eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0200072710 tx=0x7f01f80104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:59.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.999+0000 7f01e77fe700 1 -- 172.21.15.89:0/448470831 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f801b070 con 0x7f02000c8a40 2023-07-06T02:23:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.999+0000 7f01e77fe700 1 -- 172.21.15.89:0/448470831 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f800f040 con 0x7f02000c8a40 2023-07-06T02:23:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.999+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02000c9d70 con 0x7f02000c8a40 2023-07-06T02:23:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.999+0000 7f01e77fe700 1 -- 172.21.15.89:0/448470831 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800e050 con 0x7f02000c8a40 2023-07-06T02:23:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:58.999+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02000c93f0 con 0x7f02000c8a40 2023-07-06T02:23:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.000+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0200073140 con 0x7f02000c8a40 2023-07-06T02:23:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.000+0000 7f01e77fe700 1 -- 172.21.15.89:0/448470831 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f01f8015440 con 0x7f02000c8a40 2023-07-06T02:23:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.001+0000 7f01e77fe700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01ec06ace0 0x7f01ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:23:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.001+0000 7f01e77fe700 1 -- 172.21.15.89:0/448470831 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f01f8089810 con 0x7f02000c8a40 2023-07-06T02:23:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.001+0000 7f01fed9d700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01ec06ace0 0x7f01ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:23:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.001+0000 7f01fed9d700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01ec06ace0 0x7f01ec06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01f4015040 tx=0x7f01f4005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:23:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.005+0000 7f01e77fe700 1 -- 172.21.15.89:0/448470831 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0200073140 con 0x7f02000c8a40 2023-07-06T02:23:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:23:59.193+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0200073140 con 0x7f02000c8a40 2023-07-06T02:24:00.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.187+0000 7f01e77fe700 1 -- 172.21.15.89:0/448470831 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0200073140 con 0x7f02000c8a40 2023-07-06T02:24:00.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01ec06ace0 msgr2=0x7f01ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:00.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01ec06ace0 0x7f01ec06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01f4015040 tx=0x7f01f4005000 comp rx=0 tx=0).stop 2023-07-06T02:24:00.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02000c8a40 msgr2=0x7f02000c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:00.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02000c8a40 0x7f02000c8eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0200072710 tx=0x7f01f80104a0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 shutdown_connections 2023-07-06T02:24:00.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01ec06ace0 0x7f01ec06d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02000c8a40 0x7f02000c8eb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01e8003a00 0x7f02000c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 --2- 172.21.15.89:0/448470831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020014bc70 0x7f020014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 >> 172.21.15.89:0/448470831 conn(0x7f02000cd5c0 msgr2=0x7f0200157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:00.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 shutdown_connections 2023-07-06T02:24:00.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.190+0000 7f0205220700 1 -- 172.21.15.89:0/448470831 wait complete. 2023-07-06T02:24:00.195 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-07-06T02:24:00.209 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T02:24:00.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.335+0000 7f01da77b700 1 Processor -- start 2023-07-06T02:24:00.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.335+0000 7f01da77b700 1 -- start start 2023-07-06T02:24:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01da77b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01d4156c30 0x7f01d40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01da77b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d406e720 con 0x7f01d4071670 2023-07-06T02:24:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01da77b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d4157000 con 0x7f01d4071af0 2023-07-06T02:24:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01da77b700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d41602c0 con 0x7f01d4156c30 2023-07-06T02:24:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01d3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01d4156c30 0x7f01d40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01d37fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f01d4071670 0x7f01d406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59330/0 (socket says 172.21.15.89:59330) 2023-07-06T02:24:00.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01d3fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f01d4156c30 0x7f01d40cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34424/0 (socket says 172.21.15.89:34424) 2023-07-06T02:24:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.336+0000 7f01d37fe700 1 -- 172.21.15.89:0/3451098273 learned_addr learned my addr 172.21.15.89:0/3451098273 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d3fff700 1 -- 172.21.15.89:0/3451098273 >> v1:172.21.15.172:6789/0 conn(0x7f01d4071af0 legacy=0x7f01d406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d3fff700 1 -- 172.21.15.89:0/3451098273 >> v1:172.21.15.89:6789/0 conn(0x7f01d4071670 legacy=0x7f01d406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d3fff700 1 -- 172.21.15.89:0/3451098273 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d41605a0 con 0x7f01d4156c30 2023-07-06T02:24:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d2ffd700 1 -- 172.21.15.89:0/3451098273 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117590491 0 0) 0x7f01d406e720 con 0x7f01d4071670 2023-07-06T02:24:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d3fff700 1 --2- 172.21.15.89:0/3451098273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d40cd5f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009840 tx=0x7f01c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb1647be5b550b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:00.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d2ffd700 1 -- 172.21.15.89:0/3451098273 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8004230 con 0x7f01d4156c30 2023-07-06T02:24:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d2ffd700 1 -- 172.21.15.89:0/3451098273 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8004390 con 0x7f01d4156c30 2023-07-06T02:24:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01d2ffd700 1 -- 172.21.15.89:0/3451098273 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80036a0 con 0x7f01d4156c30 2023-07-06T02:24:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01da77b700 1 -- 172.21.15.89:0/3451098273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 msgr2=0x7f01d40cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01da77b700 1 --2- 172.21.15.89:0/3451098273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d40cd5f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009840 tx=0x7f01c8009b50 comp rx=0 tx=0).stop 2023-07-06T02:24:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01da77b700 1 -- 172.21.15.89:0/3451098273 shutdown_connections 2023-07-06T02:24:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01da77b700 1 --2- 172.21.15.89:0/3451098273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d40cd5f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01da77b700 1 -- 172.21.15.89:0/3451098273 >> 172.21.15.89:0/3451098273 conn(0x7f01d40cf970 msgr2=0x7f01d40cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01da77b700 1 -- 172.21.15.89:0/3451098273 shutdown_connections 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.337+0000 7f01da77b700 1 -- 172.21.15.89:0/3451098273 wait complete. 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 Processor -- start 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 -- start start 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d4071670 0x7f01d40af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01d4071af0 0x7f01d414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d41f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40afcc0 con 0x7f01d4156c30 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40b0610 con 0x7f01d4071670 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.339+0000 7f01da77b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40af2f0 con 0x7f01d4071af0 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d41f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d41f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34434/0 (socket says 172.21.15.89:34434) 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 -- 172.21.15.89:0/2068576098 learned_addr learned my addr 172.21.15.89:0/2068576098 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d3fff700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d4071670 0x7f01d40af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d37fe700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01d4071af0 0x7f01d414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 -- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01d4071af0 msgr2=0x7f01d414ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01d4071af0 0x7f01d414ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 -- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d4071670 msgr2=0x7f01d40af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d4071670 0x7f01d40af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 -- 172.21.15.89:0/2068576098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80094a0 con 0x7f01d4156c30 2023-07-06T02:24:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d3fff700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d4071670 0x7f01d40af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d8d18700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d41f5570 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00cc10 tx=0x7f01cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d0ff9700 1 -- 172.21.15.89:0/2068576098 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00abb0 con 0x7f01d4156c30 2023-07-06T02:24:00.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d0ff9700 1 -- 172.21.15.89:0/2068576098 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc00ad10 con 0x7f01d4156c30 2023-07-06T02:24:00.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d414d3b0 con 0x7f01d4156c30 2023-07-06T02:24:00.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01d0ff9700 1 -- 172.21.15.89:0/2068576098 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc012660 con 0x7f01d4156c30 2023-07-06T02:24:00.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.340+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d41f6460 con 0x7f01d4156c30 2023-07-06T02:24:00.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.341+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d40af460 con 0x7f01d4156c30 2023-07-06T02:24:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.342+0000 7f01d0ff9700 1 -- 172.21.15.89:0/2068576098 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f01cc0127c0 con 0x7f01d4156c30 2023-07-06T02:24:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.342+0000 7f01d0ff9700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01c006ace0 0x7f01c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.342+0000 7f01d0ff9700 1 -- 172.21.15.89:0/2068576098 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f01cc089de0 con 0x7f01d4156c30 2023-07-06T02:24:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.342+0000 7f01d3fff700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01c006ace0 0x7f01c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.342+0000 7f01d3fff700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01c006ace0 0x7f01c006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f01c80090b0 tx=0x7f01c8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.346+0000 7f01d0ff9700 1 -- 172.21.15.89:0/2068576098 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f01cc054cc0 con 0x7f01d4156c30 2023-07-06T02:24:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:00.538+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01d40742e0 con 0x7f01d4156c30 2023-07-06T02:24:01.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.463+0000 7f01d0ff9700 1 -- 172.21.15.89:0/2068576098 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f01cc016020 con 0x7f01d4156c30 2023-07-06T02:24:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01c006ace0 msgr2=0x7f01c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01c006ace0 0x7f01c006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f01c80090b0 tx=0x7f01c8008030 comp rx=0 tx=0).stop 2023-07-06T02:24:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 msgr2=0x7f01d41f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:01.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d41f5570 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00cc10 tx=0x7f01cc00a060 comp rx=0 tx=0).stop 2023-07-06T02:24:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 shutdown_connections 2023-07-06T02:24:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f01c006ace0 0x7f01c006d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01d4156c30 0x7f01d41f5570 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d4071670 0x7f01d40af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 --2- 172.21.15.89:0/2068576098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01d4071af0 0x7f01d414ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 >> 172.21.15.89:0/2068576098 conn(0x7f01d40cf970 msgr2=0x7f01d4152f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:01.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 shutdown_connections 2023-07-06T02:24:01.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.469+0000 7f01da77b700 1 -- 172.21.15.89:0/2068576098 wait complete. 2023-07-06T02:24:01.474 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-07-06T02:24:01.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-06T02:24:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.618+0000 7f47cc17e700 1 Processor -- start 2023-07-06T02:24:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.619+0000 7f47cc17e700 1 -- start start 2023-07-06T02:24:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.619+0000 7f47cc17e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47c4156c30 0x7f47c40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.619+0000 7f47cc17e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47c4071670 0x7f47c4071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47cc17e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c4072030 con 0x7f47c406d900 2023-07-06T02:24:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47cc17e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c4157000 con 0x7f47c4071670 2023-07-06T02:24:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47cc17e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c41602c0 con 0x7f47c4156c30 2023-07-06T02:24:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47ca71b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f47c406d900 0x7f47c406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59340/0 (socket says 172.21.15.89:59340) 2023-07-06T02:24:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47ca71b700 1 -- 172.21.15.89:0/535902056 learned_addr learned my addr 172.21.15.89:0/535902056 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:01.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c9719700 1 --2- 172.21.15.89:0/535902056 >> v2:172.21.15.89:3300/0 conn(0x7f47c4071670 0x7f47c4071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c9f1a700 1 --2- 172.21.15.89:0/535902056 >> v2:172.21.15.172:3300/0 conn(0x7f47c4156c30 0x7f47c40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c8f18700 1 -- 172.21.15.89:0/535902056 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504334607 0 0) 0x7f47c4072030 con 0x7f47c406d900 2023-07-06T02:24:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c8f18700 1 -- 172.21.15.89:0/535902056 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b00035b0 con 0x7f47c406d900 2023-07-06T02:24:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c9719700 1 -- 172.21.15.89:0/535902056 >> v2:172.21.15.172:3300/0 conn(0x7f47c4156c30 msgr2=0x7f47c40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c9719700 1 --2- 172.21.15.89:0/535902056 >> v2:172.21.15.172:3300/0 conn(0x7f47c4156c30 0x7f47c40cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c9719700 1 -- 172.21.15.89:0/535902056 >> v1:172.21.15.89:6789/0 conn(0x7f47c406d900 legacy=0x7f47c406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c9719700 1 -- 172.21.15.89:0/535902056 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c41605a0 con 0x7f47c4071670 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.620+0000 7f47c9f1a700 1 --2- 172.21.15.89:0/535902056 >> v2:172.21.15.172:3300/0 conn(0x7f47c4156c30 0x7f47c40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47c9719700 1 --2- 172.21.15.89:0/535902056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c4071af0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f47c4066c10 tx=0x7f47b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d013ba43be07c7ef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47c8f18700 1 -- 172.21.15.89:0/535902056 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4011830 con 0x7f47c4071670 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47c8f18700 1 -- 172.21.15.89:0/535902056 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4011990 con 0x7f47c4071670 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47c8f18700 1 -- 172.21.15.89:0/535902056 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4011d00 con 0x7f47c4071670 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 -- 172.21.15.89:0/535902056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 msgr2=0x7f47c4071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 --2- 172.21.15.89:0/535902056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c4071af0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f47c4066c10 tx=0x7f47b4009b20 comp rx=0 tx=0).stop 2023-07-06T02:24:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 -- 172.21.15.89:0/535902056 shutdown_connections 2023-07-06T02:24:01.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 --2- 172.21.15.89:0/535902056 >> v2:172.21.15.172:3300/0 conn(0x7f47c4156c30 0x7f47c40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 --2- 172.21.15.89:0/535902056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c4071af0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 -- 172.21.15.89:0/535902056 >> 172.21.15.89:0/535902056 conn(0x7f47c40cf970 msgr2=0x7f47c40cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:01.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 -- 172.21.15.89:0/535902056 shutdown_connections 2023-07-06T02:24:01.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.621+0000 7f47cc17e700 1 -- 172.21.15.89:0/535902056 wait complete. 2023-07-06T02:24:01.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.623+0000 7f47cc17e700 1 Processor -- start 2023-07-06T02:24:01.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.623+0000 7f47cc17e700 1 -- start start 2023-07-06T02:24:01.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406d900 0x7f47c41f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c4156c30 0x7f47c414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41f2ed0 con 0x7f47c4071670 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40b2be0 con 0x7f47c4156c30 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41f66c0 con 0x7f47c406d900 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34446/0 (socket says 172.21.15.89:34446) 2023-07-06T02:24:01.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 -- 172.21.15.89:0/761886201 learned_addr learned my addr 172.21.15.89:0/761886201 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:01.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9f1a700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406d900 0x7f47c41f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:01.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47ca71b700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c4156c30 0x7f47c414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:01.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 -- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406d900 msgr2=0x7f47c41f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:01.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406d900 0x7f47c41f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 -- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c4156c30 msgr2=0x7f47c414ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:01.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c4156c30 0x7f47c414ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:01.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 -- 172.21.15.89:0/761886201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40094a0 con 0x7f47c4071670 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9f1a700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406d900 0x7f47c41f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47ca71b700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c4156c30 0x7f47c414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47c9719700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c406c490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009b60 tx=0x7f47b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47ba7fc700 1 -- 172.21.15.89:0/761886201 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b40036a0 con 0x7f47c4071670 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47ba7fc700 1 -- 172.21.15.89:0/761886201 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b401f070 con 0x7f47c4071670 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c41f2290 con 0x7f47c4071670 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47ba7fc700 1 -- 172.21.15.89:0/761886201 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4003a40 con 0x7f47c4071670 2023-07-06T02:24:01.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.624+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c41f2810 con 0x7f47c4071670 2023-07-06T02:24:01.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.625+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c40742e0 con 0x7f47c4071670 2023-07-06T02:24:01.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.626+0000 7f47ba7fc700 1 -- 172.21.15.89:0/761886201 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47b4003ba0 con 0x7f47c4071670 2023-07-06T02:24:01.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.626+0000 7f47ba7fc700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f47a806ace0 0x7f47a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:01.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.626+0000 7f47ba7fc700 1 -- 172.21.15.89:0/761886201 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f47b4021070 con 0x7f47c4071670 2023-07-06T02:24:01.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.626+0000 7f47c9f1a700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f47a806ace0 0x7f47a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:01.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.627+0000 7f47c9f1a700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f47a806ace0 0x7f47a806d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f47c000c960 tx=0x7f47c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:01.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.631+0000 7f47ba7fc700 1 -- 172.21.15.89:0/761886201 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f47b4055e50 con 0x7f47c4071670 2023-07-06T02:24:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:01.824+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f47c40c9fc0 con 0x7f47c4071670 2023-07-06T02:24:02.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.466+0000 7f47ba7fc700 1 -- 172.21.15.89:0/761886201 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f47b40150d0 con 0x7f47c4071670 2023-07-06T02:24:02.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f47a806ace0 msgr2=0x7f47a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:02.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f47a806ace0 0x7f47a806d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f47c000c960 tx=0x7f47c0008040 comp rx=0 tx=0).stop 2023-07-06T02:24:02.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 msgr2=0x7f47c406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:02.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c406c490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009b60 tx=0x7f47b4004750 comp rx=0 tx=0).stop 2023-07-06T02:24:02.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 shutdown_connections 2023-07-06T02:24:02.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c4156c30 0x7f47c414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:02.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406d900 0x7f47c41f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f47a806ace0 0x7f47a806d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 --2- 172.21.15.89:0/761886201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c4071670 0x7f47c406c490 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 >> 172.21.15.89:0/761886201 conn(0x7f47c40cf970 msgr2=0x7f47c415dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 shutdown_connections 2023-07-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.471+0000 7f47cc17e700 1 -- 172.21.15.89:0/761886201 wait complete. 2023-07-06T02:24:02.475 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' removed 2023-07-06T02:24:02.488 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T02:24:02.489 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T02:24:02.489 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T02:24:02.489 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T02:24:02.490 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T02:24:02.490 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T02:24:02.490 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T02:24:02.490 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T02:24:02.490 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T02:24:02.491 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T02:24:02.491 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T02:24:02.491 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T02:24:02.491 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T02:24:02.492 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T02:24:02.492 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T02:24:02.492 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T02:24:02.499 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T02:24:02.499 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T02:24:02.500 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T02:24:02.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T02:24:02.501 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T02:24:02.502 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T02:24:02.502 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T02:24:02.503 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T02:24:02.542 INFO:tasks.ceph.mds.a:Started 2023-07-06T02:24:02.542 DEBUG:teuthology.parallel:result is None 2023-07-06T02:24:02.543 INFO:tasks.ceph.mds.c:Started 2023-07-06T02:24:02.544 DEBUG:teuthology.parallel:result is None 2023-07-06T02:24:02.544 INFO:tasks.ceph.mds.b:Started 2023-07-06T02:24:02.545 DEBUG:teuthology.parallel:result is None 2023-07-06T02:24:02.545 INFO:tasks.ceph.mds.d:Started 2023-07-06T02:24:02.546 DEBUG:teuthology.parallel:result is None 2023-07-06T02:24:02.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T02:24:02.629 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-07-06T02:24:02.629 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-07-06T02:24:02.631 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-07-06T02:24:02.631 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-07-06T02:24:02.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.681+0000 7fefe9844700 1 Processor -- start 2023-07-06T02:24:02.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.681+0000 7fefe9844700 1 -- start start 2023-07-06T02:24:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.682+0000 7fefe9844700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefe40cd4d0 0x7fefe4160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.682+0000 7fefe9844700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe40cd930 con 0x7fefe406c5e0 2023-07-06T02:24:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.682+0000 7fefe9844700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe40cdb60 con 0x7fefe4159fd0 2023-07-06T02:24:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.682+0000 7fefe9844700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe415ce60 con 0x7fefe40cd4d0 2023-07-06T02:24:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.682+0000 7fefe37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefe40cd4d0 0x7fefe4160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.682+0000 7fefe27fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fefe406c5e0 0x7fefe40d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59380/0 (socket says 172.21.15.89:59380) 2023-07-06T02:24:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.682+0000 7fefe27fc700 1 -- 172.21.15.89:0/987563689 learned_addr learned my addr 172.21.15.89:0/987563689 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe37fe700 1 -- 172.21.15.89:0/987563689 >> v1:172.21.15.172:6789/0 conn(0x7fefe4159fd0 legacy=0x7fefe406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe37fe700 1 -- 172.21.15.89:0/987563689 >> v1:172.21.15.89:6789/0 conn(0x7fefe406c5e0 legacy=0x7fefe40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe1ffb700 1 -- 172.21.15.89:0/987563689 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467937680 0 0) 0x7fefe40cd930 con 0x7fefe406c5e0 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe37fe700 1 -- 172.21.15.89:0/987563689 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe415d140 con 0x7fefe40cd4d0 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe37fe700 1 --2- 172.21.15.89:0/987563689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4160b80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009d90 tx=0x7fefdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96bbd7afb1e2610 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe1ffb700 1 -- 172.21.15.89:0/987563689 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00e070 con 0x7fefe40cd4d0 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe1ffb700 1 -- 172.21.15.89:0/987563689 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc00bb90 con 0x7fefe40cd4d0 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe1ffb700 1 -- 172.21.15.89:0/987563689 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc003a20 con 0x7fefe40cd4d0 2023-07-06T02:24:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe9844700 1 -- 172.21.15.89:0/987563689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 msgr2=0x7fefe4160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe9844700 1 --2- 172.21.15.89:0/987563689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4160b80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009d90 tx=0x7fefdc00b330 comp rx=0 tx=0).stop 2023-07-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe9844700 1 -- 172.21.15.89:0/987563689 shutdown_connections 2023-07-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.683+0000 7fefe9844700 1 --2- 172.21.15.89:0/987563689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4160b80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.684+0000 7fefe9844700 1 -- 172.21.15.89:0/987563689 >> 172.21.15.89:0/987563689 conn(0x7fefe40d0590 msgr2=0x7fefe40d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.684+0000 7fefe9844700 1 -- 172.21.15.89:0/987563689 shutdown_connections 2023-07-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.684+0000 7fefe9844700 1 -- 172.21.15.89:0/987563689 wait complete. 2023-07-06T02:24:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.685+0000 7fefe9844700 1 Processor -- start 2023-07-06T02:24:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.685+0000 7fefe9844700 1 -- start start 2023-07-06T02:24:02.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe406c5e0 0x7fefe4150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefe4159fd0 0x7fefe41512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40c8700 con 0x7fefe40cd4d0 2023-07-06T02:24:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40c8b20 con 0x7fefe406c5e0 2023-07-06T02:24:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40c7a80 con 0x7fefe4159fd0 2023-07-06T02:24:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34484/0 (socket says 172.21.15.89:34484) 2023-07-06T02:24:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 -- 172.21.15.89:0/4040122544 learned_addr learned my addr 172.21.15.89:0/4040122544 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe2ffd700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe406c5e0 0x7fefe4150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe37fe700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefe4159fd0 0x7fefe41512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 -- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefe4159fd0 msgr2=0x7fefe41512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefe4159fd0 0x7fefe41512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 -- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe406c5e0 msgr2=0x7fefe4150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe406c5e0 0x7fefe4150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 -- 172.21.15.89:0/4040122544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc009a70 con 0x7fefe40cd4d0 2023-07-06T02:24:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe27fc700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4150da0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fefd40011e0 tx=0x7fefd400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefdbfff700 1 -- 172.21.15.89:0/4040122544 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4004c80 con 0x7fefe40cd4d0 2023-07-06T02:24:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefdbfff700 1 -- 172.21.15.89:0/4040122544 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4004e00 con 0x7fefe40cd4d0 2023-07-06T02:24:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe40c7d60 con 0x7fefe40cd4d0 2023-07-06T02:24:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefdbfff700 1 -- 172.21.15.89:0/4040122544 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4007310 con 0x7fefe40cd4d0 2023-07-06T02:24:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.686+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe40c8320 con 0x7fefe40cd4d0 2023-07-06T02:24:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.687+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe40c8840 con 0x7fefe40cd4d0 2023-07-06T02:24:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.688+0000 7fefdbfff700 1 -- 172.21.15.89:0/4040122544 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefd4007470 con 0x7fefe40cd4d0 2023-07-06T02:24:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.688+0000 7fefdbfff700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fefc406ace0 0x7fefc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.688+0000 7fefdbfff700 1 -- 172.21.15.89:0/4040122544 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fefd4088ee0 con 0x7fefe40cd4d0 2023-07-06T02:24:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.688+0000 7fefe2ffd700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fefc406ace0 0x7fefc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.689+0000 7fefe2ffd700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fefc406ace0 0x7fefc406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fefe415c740 tx=0x7fefcc0090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:02.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.693+0000 7fefdbfff700 1 -- 172.21.15.89:0/4040122544 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fefe40c8840 con 0x7fefe40cd4d0 2023-07-06T02:24:02.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:02.893+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fefe40c8840 con 0x7fefe40cd4d0 2023-07-06T02:24:03.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.486+0000 7fefdbfff700 1 -- 172.21.15.89:0/4040122544 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fefe40c8840 con 0x7fefe40cd4d0 2023-07-06T02:24:03.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fefc406ace0 msgr2=0x7fefc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:03.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fefc406ace0 0x7fefc406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fefe415c740 tx=0x7fefcc0090e0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 msgr2=0x7fefe4150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:03.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4150da0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fefd40011e0 tx=0x7fefd400ca30 comp rx=0 tx=0).stop 2023-07-06T02:24:03.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 shutdown_connections 2023-07-06T02:24:03.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefe4159fd0 0x7fefe41512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe406c5e0 0x7fefe4150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fefc406ace0 0x7fefc406d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 --2- 172.21.15.89:0/4040122544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefe40cd4d0 0x7fefe4150da0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.488+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 >> 172.21.15.89:0/4040122544 conn(0x7fefe40d0590 msgr2=0x7fefe415f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:03.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.489+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 shutdown_connections 2023-07-06T02:24:03.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.489+0000 7fefe9844700 1 -- 172.21.15.89:0/4040122544 wait complete. 2023-07-06T02:24:03.493 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-07-06T02:24:03.505 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T02:24:03.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.624+0000 7f6acedb1700 1 Processor -- start 2023-07-06T02:24:03.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.624+0000 7f6acedb1700 1 -- start start 2023-07-06T02:24:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6acedb1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 0x7f6ac80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6acedb1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ac80cd180 0x7f6ac815a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:03.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6acedb1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac806e110 con 0x7f6ac806dc10 2023-07-06T02:24:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6acedb1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac806e3f0 con 0x7f6ac80cd180 2023-07-06T02:24:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6acedb1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac806e6d0 con 0x7f6ac80705a0 2023-07-06T02:24:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ac80cd180 0x7f6ac815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ac80cd180 0x7f6ac815a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34494/0 (socket says 172.21.15.89:34494) 2023-07-06T02:24:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 -- 172.21.15.89:0/3160030443 learned_addr learned my addr 172.21.15.89:0/3160030443 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 -- 172.21.15.89:0/3160030443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 msgr2=0x7f6ac80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:03.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6accb4d700 1 --2- 172.21.15.89:0/3160030443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 0x7f6ac80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 --2- 172.21.15.89:0/3160030443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 0x7f6ac80ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 -- 172.21.15.89:0/3160030443 >> v1:172.21.15.172:6789/0 conn(0x7f6ac806dc10 legacy=0x7f6ac8157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 -- 172.21.15.89:0/3160030443 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac806e9b0 con 0x7f6ac80cd180 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac7fff700 1 --2- 172.21.15.89:0/3160030443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac80cd180 0x7f6ac815a870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40098f0 tx=0x7f6ab4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a5de00628e2e91 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.625+0000 7f6ac77fe700 1 -- 172.21.15.89:0/3160030443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4004230 con 0x7f6ac80cd180 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6ac77fe700 1 -- 172.21.15.89:0/3160030443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab4004390 con 0x7f6ac80cd180 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6ac77fe700 1 -- 172.21.15.89:0/3160030443 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40036a0 con 0x7f6ac80cd180 2023-07-06T02:24:03.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 -- 172.21.15.89:0/3160030443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac80cd180 msgr2=0x7f6ac815a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:03.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3160030443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac80cd180 0x7f6ac815a870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40098f0 tx=0x7f6ab4009c00 comp rx=0 tx=0).stop 2023-07-06T02:24:03.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 -- 172.21.15.89:0/3160030443 shutdown_connections 2023-07-06T02:24:03.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3160030443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 0x7f6ac80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3160030443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac80cd180 0x7f6ac815a870 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 -- 172.21.15.89:0/3160030443 >> 172.21.15.89:0/3160030443 conn(0x7f6ac80d0b00 msgr2=0x7f6ac80d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:03.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 -- 172.21.15.89:0/3160030443 shutdown_connections 2023-07-06T02:24:03.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.626+0000 7f6acedb1700 1 -- 172.21.15.89:0/3160030443 wait complete. 2023-07-06T02:24:03.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.628+0000 7f6acedb1700 1 Processor -- start 2023-07-06T02:24:03.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.628+0000 7f6acedb1700 1 -- start start 2023-07-06T02:24:03.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac806dc10 0x7f6ac80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:03.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 0x7f6ac8159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:03.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ac80cd180 0x7f6ac80c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:03.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80c87a0 con 0x7f6ac806dc10 2023-07-06T02:24:03.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80c7bd0 con 0x7f6ac80cd180 2023-07-06T02:24:03.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80c7e70 con 0x7f6ac80705a0 2023-07-06T02:24:03.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac806dc10 0x7f6ac80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:03.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac806dc10 0x7f6ac80b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34498/0 (socket says 172.21.15.89:34498) 2023-07-06T02:24:03.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 -- 172.21.15.89:0/3736388182 learned_addr learned my addr 172.21.15.89:0/3736388182 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:03.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acd34e700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ac80cd180 0x7f6ac80c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:03.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 -- 172.21.15.89:0/3736388182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 msgr2=0x7f6ac8159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:03.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 0x7f6ac8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 -- 172.21.15.89:0/3736388182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ac80cd180 msgr2=0x7f6ac80c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ac80cd180 0x7f6ac80c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 -- 172.21.15.89:0/3736388182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab4009550 con 0x7f6ac806dc10 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6accb4d700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac806dc10 0x7f6ac80b4340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c7f0 tx=0x7f6abc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6ac57fa700 1 -- 172.21.15.89:0/3736388182 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc011800 con 0x7f6ac806dc10 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6ac57fa700 1 -- 172.21.15.89:0/3736388182 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc011960 con 0x7f6ac806dc10 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6ac57fa700 1 -- 172.21.15.89:0/3736388182 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc011cf0 con 0x7f6ac806dc10 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac80c8150 con 0x7f6ac806dc10 2023-07-06T02:24:03.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.629+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac80b4880 con 0x7f6ac806dc10 2023-07-06T02:24:03.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.630+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac80c8a80 con 0x7f6ac806dc10 2023-07-06T02:24:03.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.631+0000 7f6ac57fa700 1 -- 172.21.15.89:0/3736388182 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6abc018360 con 0x7f6ac806dc10 2023-07-06T02:24:03.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.631+0000 7f6ac57fa700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6aac06ace0 0x7f6aac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:03.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.631+0000 7f6ac57fa700 1 -- 172.21.15.89:0/3736388182 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6abc0892a0 con 0x7f6ac806dc10 2023-07-06T02:24:03.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.631+0000 7f6ac7fff700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6aac06ace0 0x7f6aac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:03.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.632+0000 7f6ac7fff700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6aac06ace0 0x7f6aac06d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80cd550 tx=0x7f6ab4010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.636+0000 7f6ac57fa700 1 -- 172.21.15.89:0/3736388182 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6abc058000 con 0x7f6ac806dc10 2023-07-06T02:24:03.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.897+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6ac8074270 con 0x7f6ac806dc10 2023-07-06T02:24:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.898+0000 7f6ac57fa700 1 -- 172.21.15.89:0/3736388182 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f6abc057b90 con 0x7f6ac806dc10 2023-07-06T02:24:03.901 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:03.901 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-07-06T02:24:03.901 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-07-06T02:24:03.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.901 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-07-06T02:24:03.902 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBIJaZkxc0eJxAArsU7ZBxo424bxLsp4XDQzg==", 2023-07-06T02:24:03.902 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-07-06T02:24:03.902 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-07-06T02:24:03.902 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.902 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBIJaZkdkpjMxAAgucXDyMKlD5CNNhUd2FVsw==", 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-07-06T02:24:03.903 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-07-06T02:24:03.904 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.904 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.904 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-07-06T02:24:03.904 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBIJaZk4Kp8LhAA7ZzUGSx1VH4/nIry1no9mw==", 2023-07-06T02:24:03.904 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.905 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBIJaZkJZnhOhAArhNFsiedGlQACplx1ksQlA==", 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.907 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-07-06T02:24:03.907 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBMJaZkNQohGBAAZP652zOWUgQp13HnR6+F8A==", 2023-07-06T02:24:03.907 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.907 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.907 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.907 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBPJaZksNJiGBAARGc+H97BjsQQDZN7ecj0hg==", 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBSJaZkCekLGRAAr8LzcL/AGb2ZoNpXqM2eWA==", 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.909 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBVJaZkCCbTGBAAx7qRQFCR8YGiZVPXE1zI2g==", 2023-07-06T02:24:03.910 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.911 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBbJaZks3RsHRAAMWA/q3s6vfs4F5bDUN0MlA==", 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBeJaZkE8vXHBAAXrYe0Q68x+YFaHH2u1byjg==", 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBhJaZkpgJ1HBAA/FE2+xljcEhieO2g7VmuTg==", 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.915 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-07-06T02:24:03.915 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBkJaZkdUl6GxAAz4jyZZYYmRikOz52+h97DA==", 2023-07-06T02:24:03.915 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.915 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-07-06T02:24:03.915 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJJaZkfY62AxAAWC7BJI+7fFi+ew8E4NJVSA==", 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.916 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-07-06T02:24:03.917 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJJaZkqSXYDBAAadl28XEDPg/BP/C0riEa6w==", 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJJaZkf3VxCRAA8VDNglGJ+NZaFkDTckVb+Q==", 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-07-06T02:24:03.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.920 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-07-06T02:24:03.920 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJJaZkGdOoEhAAaC9UVh7c1lNsWOPavhp0Xw==", 2023-07-06T02:24:03.920 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.920 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-07-06T02:24:03.920 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg==", 2023-07-06T02:24:03.921 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.922 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-mds", 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBqJaZkVyi0GBAAKi7OYCAJGe7Q7T5FclWXpA==", 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-mds" 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-mgr", 2023-07-06T02:24:03.923 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBqJaZkmjC0GBAA0+t34tyPtw43KVSX+Jl8ig==", 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-osd", 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBqJaZkQTm0GBAAJ3UN7qmAsuQN5abLDAdl9g==", 2023-07-06T02:24:03.924 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-osd" 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rbd", 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBqJaZkT0C0GBAAyJjmPjgU/L6HZUltoRC0Gw==", 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.925 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-06T02:24:03.926 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.926 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-06T02:24:03.926 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBqJaZkoEe0GBAAJa41E4Zddxg1SqW+cIMAFA==", 2023-07-06T02:24:03.926 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.926 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rgw", 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBqJaZkDU+0GBAA2c1VRhgnvnSOUmrn3O3/8Q==", 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-06T02:24:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBIJaZk+NmwIxAA8izORTN1WAcvae0VX/dzlg==", 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-07-06T02:24:03.928 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBIJaZkEdagHxAA9/2TvHUk+ijKGs7jqWs4Uw==", 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-07-06T02:24:03.929 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-07-06T02:24:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-07-06T02:24:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-07-06T02:24:03.930 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-07-06T02:24:03.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.899+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6aac06ace0 msgr2=0x7f6aac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.899+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6aac06ace0 0x7f6aac06d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80cd550 tx=0x7f6ab4010000 comp rx=0 tx=0).stop 2023-07-06T02:24:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.899+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac806dc10 msgr2=0x7f6ac80b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.899+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac806dc10 0x7f6ac80b4340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c7f0 tx=0x7f6abc00cbb0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 shutdown_connections 2023-07-06T02:24:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ac80cd180 0x7f6ac80c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6aac06ace0 0x7f6aac06d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ac806dc10 0x7f6ac80b4340 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 --2- 172.21.15.89:0/3736388182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ac80705a0 0x7f6ac8159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 >> 172.21.15.89:0/3736388182 conn(0x7f6ac80d0b00 msgr2=0x7f6ac815c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 shutdown_connections 2023-07-06T02:24:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:03.900+0000 7f6acedb1700 1 -- 172.21.15.89:0/3736388182 wait complete. 2023-07-06T02:24:03.933 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-06T02:24:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.052+0000 7f7f6c483700 1 Processor -- start 2023-07-06T02:24:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.052+0000 7f7f6c483700 1 -- start start 2023-07-06T02:24:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6c483700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f641600b0 0x7f7f640b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6c483700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f640cb3d0 con 0x7f7f640b19d0 2023-07-06T02:24:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6c483700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f640b1dd0 con 0x7f7f64156ab0 2023-07-06T02:24:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6c483700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f64156e80 con 0x7f7f641600b0 2023-07-06T02:24:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f69a1e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f641600b0 0x7f7f640b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6aa20700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7f640b19d0 0x7f7f640cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59412/0 (socket says 172.21.15.89:59412) 2023-07-06T02:24:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f69a1e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f641600b0 0x7f7f640b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34508/0 (socket says 172.21.15.89:34508) 2023-07-06T02:24:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6aa20700 1 -- 172.21.15.89:0/1966034109 learned_addr learned my addr 172.21.15.89:0/1966034109 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f69a1e700 1 -- 172.21.15.89:0/1966034109 >> v1:172.21.15.172:6789/0 conn(0x7f7f64156ab0 legacy=0x7f7f64160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f69a1e700 1 -- 172.21.15.89:0/1966034109 >> v1:172.21.15.89:6789/0 conn(0x7f7f640b19d0 legacy=0x7f7f640cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f69a1e700 1 -- 172.21.15.89:0/1966034109 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640ca370 con 0x7f7f641600b0 2023-07-06T02:24:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6921d700 1 -- 172.21.15.89:0/1966034109 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857086168 0 0) 0x7f7f640cb3d0 con 0x7f7f640b19d0 2023-07-06T02:24:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f69a1e700 1 --2- 172.21.15.89:0/1966034109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f640b1490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009840 tx=0x7f7f54009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743165a45d4c4e7b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6921d700 1 -- 172.21.15.89:0/1966034109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54011830 con 0x7f7f641600b0 2023-07-06T02:24:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6921d700 1 -- 172.21.15.89:0/1966034109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f54011990 con 0x7f7f641600b0 2023-07-06T02:24:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.053+0000 7f7f6921d700 1 -- 172.21.15.89:0/1966034109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54011d20 con 0x7f7f641600b0 2023-07-06T02:24:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.054+0000 7f7f6c483700 1 -- 172.21.15.89:0/1966034109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 msgr2=0x7f7f640b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.054+0000 7f7f6c483700 1 --2- 172.21.15.89:0/1966034109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f640b1490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009840 tx=0x7f7f54009b50 comp rx=0 tx=0).stop 2023-07-06T02:24:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.054+0000 7f7f6c483700 1 -- 172.21.15.89:0/1966034109 shutdown_connections 2023-07-06T02:24:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.054+0000 7f7f6c483700 1 --2- 172.21.15.89:0/1966034109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f640b1490 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.054+0000 7f7f6c483700 1 -- 172.21.15.89:0/1966034109 >> 172.21.15.89:0/1966034109 conn(0x7f7f640cc490 msgr2=0x7f7f640cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.054+0000 7f7f6c483700 1 -- 172.21.15.89:0/1966034109 shutdown_connections 2023-07-06T02:24:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.054+0000 7f7f6c483700 1 -- 172.21.15.89:0/1966034109 wait complete. 2023-07-06T02:24:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.055+0000 7f7f6c483700 1 Processor -- start 2023-07-06T02:24:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.055+0000 7f7f6c483700 1 -- start start 2023-07-06T02:24:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6c483700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f640b19d0 0x7f7f6414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6c483700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f64156ab0 0x7f7f640c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6c483700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f641f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6c483700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6406ae50 con 0x7f7f641600b0 2023-07-06T02:24:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6c483700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640ae760 con 0x7f7f640b19d0 2023-07-06T02:24:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6c483700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640adee0 con 0x7f7f64156ab0 2023-07-06T02:24:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f641f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f641f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34516/0 (socket says 172.21.15.89:34516) 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 -- 172.21.15.89:0/4007546870 learned_addr learned my addr 172.21.15.89:0/4007546870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6a21f700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f640b19d0 0x7f7f6414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f69a1e700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f64156ab0 0x7f7f640c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 -- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f64156ab0 msgr2=0x7f7f640c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f64156ab0 0x7f7f640c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 -- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f640b19d0 msgr2=0x7f7f6414fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f640b19d0 0x7f7f6414fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 -- 172.21.15.89:0/4007546870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540094a0 con 0x7f7f641600b0 2023-07-06T02:24:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.056+0000 7f7f6aa20700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f641f1f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00cca0 tx=0x7f7f5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.057+0000 7f7f5affd700 1 -- 172.21.15.89:0/4007546870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00cea0 con 0x7f7f641600b0 2023-07-06T02:24:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.057+0000 7f7f5affd700 1 -- 172.21.15.89:0/4007546870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00ade0 con 0x7f7f641600b0 2023-07-06T02:24:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.057+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f640ae1a0 con 0x7f7f641600b0 2023-07-06T02:24:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.057+0000 7f7f5affd700 1 -- 172.21.15.89:0/4007546870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c007310 con 0x7f7f641600b0 2023-07-06T02:24:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.057+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f64150360 con 0x7f7f641600b0 2023-07-06T02:24:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.058+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6406afc0 con 0x7f7f641600b0 2023-07-06T02:24:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.058+0000 7f7f5affd700 1 -- 172.21.15.89:0/4007546870 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f5c007470 con 0x7f7f641600b0 2023-07-06T02:24:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.058+0000 7f7f5affd700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7f4806ace0 0x7f7f4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.058+0000 7f7f5affd700 1 -- 172.21.15.89:0/4007546870 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f7f5c089130 con 0x7f7f641600b0 2023-07-06T02:24:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.058+0000 7f7f6a21f700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7f4806ace0 0x7f7f4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.059+0000 7f7f6a21f700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7f4806ace0 0x7f7f4806d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000f60 tx=0x7f7f60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.063+0000 7f7f5affd700 1 -- 172.21.15.89:0/4007546870 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f5c054480 con 0x7f7f641600b0 2023-07-06T02:24:04.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.328+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7f64074270 con 0x7f7f641600b0 2023-07-06T02:24:04.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.334+0000 7f7f5affd700 1 -- 172.21.15.89:0/4007546870 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7f5c057aa0 con 0x7f7f641600b0 2023-07-06T02:24:04.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7f4806ace0 msgr2=0x7f7f4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7f4806ace0 0x7f7f4806d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000f60 tx=0x7f7f60009040 comp rx=0 tx=0).stop 2023-07-06T02:24:04.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 msgr2=0x7f7f641f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f641f1f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00cca0 tx=0x7f7f5c00a340 comp rx=0 tx=0).stop 2023-07-06T02:24:04.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 shutdown_connections 2023-07-06T02:24:04.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7f4806ace0 0x7f7f4806d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f641600b0 0x7f7f641f1f70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f640b19d0 0x7f7f6414fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 --2- 172.21.15.89:0/4007546870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f64156ab0 0x7f7f640c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 >> 172.21.15.89:0/4007546870 conn(0x7f7f640cc490 msgr2=0x7f7f6415f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:04.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 shutdown_connections 2023-07-06T02:24:04.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.336+0000 7f7f6c483700 1 -- 172.21.15.89:0/4007546870 wait complete. 2023-07-06T02:24:04.351 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-06T02:24:04.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.469+0000 7fcac65ef700 1 Processor -- start 2023-07-06T02:24:04.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.470+0000 7fcac65ef700 1 -- start start 2023-07-06T02:24:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.470+0000 7fcac65ef700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcac00cecf0 0x7fcac00ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.470+0000 7fcac65ef700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac00ca5f0 0x7fcac006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.470+0000 7fcac65ef700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac006ebe0 con 0x7fcac006e250 2023-07-06T02:24:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.470+0000 7fcac65ef700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac00caa30 con 0x7fcac00cecf0 2023-07-06T02:24:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.470+0000 7fcac65ef700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac006abe0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcac00cecf0 0x7fcac00ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcac00cecf0 0x7fcac00ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34520/0 (socket says 172.21.15.89:34520) 2023-07-06T02:24:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 -- 172.21.15.89:0/1071112527 learned_addr learned my addr 172.21.15.89:0/1071112527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcac4b8c700 1 --2- 172.21.15.89:0/1071112527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac00ca5f0 0x7fcac006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 -- 172.21.15.89:0/1071112527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac00ca5f0 msgr2=0x7fcac006e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 --2- 172.21.15.89:0/1071112527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac00ca5f0 0x7fcac006e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 -- 172.21.15.89:0/1071112527 >> v1:172.21.15.89:6789/0 conn(0x7fcac006e250 legacy=0x7fcac00c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 -- 172.21.15.89:0/1071112527 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac006b160 con 0x7fcac00cecf0 2023-07-06T02:24:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcac4b8c700 1 --2- 172.21.15.89:0/1071112527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac00ca5f0 0x7fcac006e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabeffd700 1 -- 172.21.15.89:0/1071112527 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276288901 0 0) 0x7fcab0001110 con 0x7fcac006e250 2023-07-06T02:24:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabffff700 1 --2- 172.21.15.89:0/1071112527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00cecf0 0x7fcac00ccd90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcab40098f0 tx=0x7fcab4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f7c906cd7f0d50 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabeffd700 1 -- 172.21.15.89:0/1071112527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab400b8a0 con 0x7fcac00cecf0 2023-07-06T02:24:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabeffd700 1 -- 172.21.15.89:0/1071112527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab400ba00 con 0x7fcac00cecf0 2023-07-06T02:24:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.471+0000 7fcabeffd700 1 -- 172.21.15.89:0/1071112527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab400bd90 con 0x7fcac00cecf0 2023-07-06T02:24:04.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 -- 172.21.15.89:0/1071112527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00cecf0 msgr2=0x7fcac00ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 --2- 172.21.15.89:0/1071112527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00cecf0 0x7fcac00ccd90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcab40098f0 tx=0x7fcab4009c00 comp rx=0 tx=0).stop 2023-07-06T02:24:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 -- 172.21.15.89:0/1071112527 shutdown_connections 2023-07-06T02:24:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 --2- 172.21.15.89:0/1071112527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac00ca5f0 0x7fcac006e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 --2- 172.21.15.89:0/1071112527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00cecf0 0x7fcac00ccd90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 -- 172.21.15.89:0/1071112527 >> 172.21.15.89:0/1071112527 conn(0x7fcac00cf1c0 msgr2=0x7fcac00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 -- 172.21.15.89:0/1071112527 shutdown_connections 2023-07-06T02:24:04.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.472+0000 7fcac65ef700 1 -- 172.21.15.89:0/1071112527 wait complete. 2023-07-06T02:24:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.473+0000 7fcac65ef700 1 Processor -- start 2023-07-06T02:24:04.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.473+0000 7fcac65ef700 1 -- start start 2023-07-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcac65ef700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac006e250 0x7fcac0152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcac65ef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00ca5f0 0x7fcac00c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcac65ef700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac00cecf0 0x7fcac0155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcac65ef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01e27d0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcac65ef700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01e2a70 con 0x7fcac00cecf0 2023-07-06T02:24:04.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcac65ef700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01e2d50 con 0x7fcac006e250 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00ca5f0 0x7fcac00c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00ca5f0 0x7fcac00c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34526/0 (socket says 172.21.15.89:34526) 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 -- 172.21.15.89:0/532795071 learned_addr learned my addr 172.21.15.89:0/532795071 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabffff700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac006e250 0x7fcac0152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcac4b8c700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac00cecf0 0x7fcac0155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 -- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac006e250 msgr2=0x7fcac0152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac006e250 0x7fcac0152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 -- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac00cecf0 msgr2=0x7fcac0155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac00cecf0 0x7fcac0155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 -- 172.21.15.89:0/532795071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4009550 con 0x7fcac00ca5f0 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.474+0000 7fcabf7fe700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00ca5f0 0x7fcac00c42d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcab000c750 tx=0x7fcab000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.475+0000 7fcabcff9700 1 -- 172.21.15.89:0/532795071 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab00054e0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.475+0000 7fcabcff9700 1 -- 172.21.15.89:0/532795071 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0003680 con 0x7fcac00ca5f0 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.475+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac01e3030 con 0x7fcac00ca5f0 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.475+0000 7fcabcff9700 1 -- 172.21.15.89:0/532795071 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0003a30 con 0x7fcac00ca5f0 2023-07-06T02:24:04.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.475+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01e35f0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.476+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac01e37e0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.476+0000 7fcabcff9700 1 -- 172.21.15.89:0/532795071 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcab0003b90 con 0x7fcac00ca5f0 2023-07-06T02:24:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.476+0000 7fcabcff9700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcaac06ace0 0x7fcaac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.477+0000 7fcabcff9700 1 -- 172.21.15.89:0/532795071 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcab00884c0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.477+0000 7fcabffff700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcaac06ace0 0x7fcaac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.477+0000 7fcabffff700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcaac06ace0 0x7fcaac06d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcab4014040 tx=0x7fcab4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.481+0000 7fcabcff9700 1 -- 172.21.15.89:0/532795071 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcab0053810 con 0x7fcac00ca5f0 2023-07-06T02:24:04.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.748+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fcac006ebe0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.752+0000 7fcabcff9700 1 -- 172.21.15.89:0/532795071 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcac006ebe0 con 0x7fcac00ca5f0 2023-07-06T02:24:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcaac06ace0 msgr2=0x7fcaac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcaac06ace0 0x7fcaac06d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcab4014040 tx=0x7fcab4008030 comp rx=0 tx=0).stop 2023-07-06T02:24:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00ca5f0 msgr2=0x7fcac00c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00ca5f0 0x7fcac00c42d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcab000c750 tx=0x7fcab000ca60 comp rx=0 tx=0).stop 2023-07-06T02:24:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 shutdown_connections 2023-07-06T02:24:04.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac00cecf0 0x7fcac0155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcac006e250 0x7fcac0152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fcaac06ace0 0x7fcaac06d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 --2- 172.21.15.89:0/532795071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcac00ca5f0 0x7fcac00c42d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 >> 172.21.15.89:0/532795071 conn(0x7fcac00cf1c0 msgr2=0x7fcac0158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:04.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 shutdown_connections 2023-07-06T02:24:04.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.754+0000 7fcac65ef700 1 -- 172.21.15.89:0/532795071 wait complete. 2023-07-06T02:24:04.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-06T02:24:04.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.886+0000 7f3350dcf700 1 Processor -- start 2023-07-06T02:24:04.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.886+0000 7f3350dcf700 1 -- start start 2023-07-06T02:24:04.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f3350dcf700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f334c156ab0 0x7f334c160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f3350dcf700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f334c1600b0 0x7f334c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f3350dcf700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0cb3d0 con 0x7f334c160bb0 2023-07-06T02:24:04.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f3350dcf700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c156e80 con 0x7f334c156ab0 2023-07-06T02:24:04.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f3350dcf700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0ca370 con 0x7f334c1600b0 2023-07-06T02:24:04.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f334c156ab0 0x7f334c160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334a59c700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f334c156ab0 0x7f334c160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34534/0 (socket says 172.21.15.89:34534) 2023-07-06T02:24:04.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334a59c700 1 -- 172.21.15.89:0/3739693057 learned_addr learned my addr 172.21.15.89:0/3739693057 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:04.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334a59c700 1 -- 172.21.15.89:0/3739693057 >> v2:172.21.15.172:3300/0 conn(0x7f334c1600b0 msgr2=0x7f334c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:04.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334ad9d700 1 --2- 172.21.15.89:0/3739693057 >> v2:172.21.15.172:3300/0 conn(0x7f334c1600b0 0x7f334c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334a59c700 1 --2- 172.21.15.89:0/3739693057 >> v2:172.21.15.172:3300/0 conn(0x7f334c1600b0 0x7f334c0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334a59c700 1 -- 172.21.15.89:0/3739693057 >> v1:172.21.15.172:6789/0 conn(0x7f334c160bb0 legacy=0x7f334c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:04.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.887+0000 7f334a59c700 1 -- 172.21.15.89:0/3739693057 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0ca610 con 0x7f334c156ab0 2023-07-06T02:24:04.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f334a59c700 1 --2- 172.21.15.89:0/3739693057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c156ab0 0x7f334c160670 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f334c06d250 tx=0x7f3340009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc86298a460d022b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f334959a700 1 -- 172.21.15.89:0/3739693057 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340004230 con 0x7f334c156ab0 2023-07-06T02:24:04.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f334959a700 1 -- 172.21.15.89:0/3739693057 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3340004390 con 0x7f334c156ab0 2023-07-06T02:24:04.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f334959a700 1 -- 172.21.15.89:0/3739693057 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400036a0 con 0x7f334c156ab0 2023-07-06T02:24:04.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 -- 172.21.15.89:0/3739693057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c156ab0 msgr2=0x7f334c160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 --2- 172.21.15.89:0/3739693057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c156ab0 0x7f334c160670 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f334c06d250 tx=0x7f3340009bd0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 -- 172.21.15.89:0/3739693057 shutdown_connections 2023-07-06T02:24:04.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 --2- 172.21.15.89:0/3739693057 >> v2:172.21.15.172:3300/0 conn(0x7f334c1600b0 0x7f334c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 --2- 172.21.15.89:0/3739693057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c156ab0 0x7f334c160670 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 -- 172.21.15.89:0/3739693057 >> 172.21.15.89:0/3739693057 conn(0x7f334c0cc490 msgr2=0x7f334c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:04.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 -- 172.21.15.89:0/3739693057 shutdown_connections 2023-07-06T02:24:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.888+0000 7f3350dcf700 1 -- 172.21.15.89:0/3739693057 wait complete. 2023-07-06T02:24:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.890+0000 7f3350dcf700 1 Processor -- start 2023-07-06T02:24:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.890+0000 7f3350dcf700 1 -- start start 2023-07-06T02:24:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f334c156ab0 0x7f334c1e7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c1600b0 0x7f334c0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f334c1e5dd0 0x7f334c142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1e6290 con 0x7f334c1600b0 2023-07-06T02:24:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1e6570 con 0x7f334c156ab0 2023-07-06T02:24:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1e6850 con 0x7f334c1e5dd0 2023-07-06T02:24:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c1600b0 0x7f334c0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c1600b0 0x7f334c0c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34542/0 (socket says 172.21.15.89:34542) 2023-07-06T02:24:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 -- 172.21.15.89:0/381714991 learned_addr learned my addr 172.21.15.89:0/381714991 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f334a59c700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f334c156ab0 0x7f334c1e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f334ad9d700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f334c1e5dd0 0x7f334c142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 -- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f334c1e5dd0 msgr2=0x7f334c142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f334c1e5dd0 0x7f334c142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 -- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f334c156ab0 msgr2=0x7f334c1e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f334c156ab0 0x7f334c1e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 -- 172.21.15.89:0/381714991 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340009550 con 0x7f334c1600b0 2023-07-06T02:24:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3349d9b700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c1600b0 0x7f334c0c59c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c4a0 tx=0x7f333c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f333affd700 1 -- 172.21.15.89:0/381714991 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c0054e0 con 0x7f334c1600b0 2023-07-06T02:24:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f333affd700 1 -- 172.21.15.89:0/381714991 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c003680 con 0x7f334c1600b0 2023-07-06T02:24:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c1e6b30 con 0x7f334c1600b0 2023-07-06T02:24:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f333affd700 1 -- 172.21.15.89:0/381714991 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c003a10 con 0x7f334c1600b0 2023-07-06T02:24:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.891+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c0c5f00 con 0x7f334c1600b0 2023-07-06T02:24:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.892+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c0d08b0 con 0x7f334c1600b0 2023-07-06T02:24:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.893+0000 7f333affd700 1 -- 172.21.15.89:0/381714991 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f333c003b70 con 0x7f334c1600b0 2023-07-06T02:24:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.893+0000 7f333affd700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f332c06ace0 0x7f332c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.893+0000 7f334a59c700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f332c06ace0 0x7f332c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:04.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.893+0000 7f333affd700 1 -- 172.21.15.89:0/381714991 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f333c088540 con 0x7f334c1600b0 2023-07-06T02:24:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.894+0000 7f334a59c700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f332c06ace0 0x7f332c06d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3340009130 tx=0x7f3340008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:04.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:04.898+0000 7f333affd700 1 -- 172.21.15.89:0/381714991 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f333c0538b0 con 0x7f334c1600b0 2023-07-06T02:24:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.159+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f334c074270 con 0x7f334c1600b0 2023-07-06T02:24:05.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.165+0000 7f333affd700 1 -- 172.21.15.89:0/381714991 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f333c056ed0 con 0x7f334c1600b0 2023-07-06T02:24:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f332c06ace0 msgr2=0x7f332c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f332c06ace0 0x7f332c06d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3340009130 tx=0x7f3340008030 comp rx=0 tx=0).stop 2023-07-06T02:24:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c1600b0 msgr2=0x7f334c0c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c1600b0 0x7f334c0c59c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c4a0 tx=0x7f333c00c7b0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 shutdown_connections 2023-07-06T02:24:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f334c1e5dd0 0x7f334c142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f334c156ab0 0x7f334c1e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f332c06ace0 0x7f332c06d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 --2- 172.21.15.89:0/381714991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f334c1600b0 0x7f334c0c59c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 >> 172.21.15.89:0/381714991 conn(0x7f334c0cc490 msgr2=0x7f334c15e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 shutdown_connections 2023-07-06T02:24:05.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.167+0000 7f3350dcf700 1 -- 172.21.15.89:0/381714991 wait complete. 2023-07-06T02:24:05.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-06T02:24:05.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.300+0000 7f95b3c62700 1 Processor -- start 2023-07-06T02:24:05.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.300+0000 7f95b3c62700 1 -- start start 2023-07-06T02:24:05.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b3c62700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95ac158960 0x7f95ac15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b3c62700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f95ac070e10 0x7f95ac071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b3c62700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac15d390 con 0x7f95ac0ccea0 2023-07-06T02:24:05.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b3c62700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac06c770 con 0x7f95ac158960 2023-07-06T02:24:05.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b3c62700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac06d5d0 con 0x7f95ac070e10 2023-07-06T02:24:05.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b19fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95ac158960 0x7f95ac15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b19fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95ac158960 0x7f95ac15ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34550/0 (socket says 172.21.15.89:34550) 2023-07-06T02:24:05.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b19fe700 1 -- 172.21.15.89:0/4241901207 learned_addr learned my addr 172.21.15.89:0/4241901207 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:05.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.301+0000 7f95b21ff700 1 --2- 172.21.15.89:0/4241901207 >> v2:172.21.15.172:3300/0 conn(0x7f95ac070e10 0x7f95ac071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b19fe700 1 -- 172.21.15.89:0/4241901207 >> v2:172.21.15.172:3300/0 conn(0x7f95ac070e10 msgr2=0x7f95ac071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:05.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b19fe700 1 --2- 172.21.15.89:0/4241901207 >> v2:172.21.15.172:3300/0 conn(0x7f95ac070e10 0x7f95ac071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b19fe700 1 -- 172.21.15.89:0/4241901207 >> v1:172.21.15.172:6789/0 conn(0x7f95ac0ccea0 legacy=0x7f95ac0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:05.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b19fe700 1 -- 172.21.15.89:0/4241901207 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac071770 con 0x7f95ac158960 2023-07-06T02:24:05.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b19fe700 1 --2- 172.21.15.89:0/4241901207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac15ce50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0ce6f0 tx=0x7f95a800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d89ffe8a7ec94f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:05.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b09fc700 1 -- 172.21.15.89:0/4241901207 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800dcf0 con 0x7f95ac158960 2023-07-06T02:24:05.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b09fc700 1 -- 172.21.15.89:0/4241901207 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800de50 con 0x7f95ac158960 2023-07-06T02:24:05.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b09fc700 1 -- 172.21.15.89:0/4241901207 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004250 con 0x7f95ac158960 2023-07-06T02:24:05.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b3c62700 1 -- 172.21.15.89:0/4241901207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 msgr2=0x7f95ac15ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b3c62700 1 --2- 172.21.15.89:0/4241901207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac15ce50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0ce6f0 tx=0x7f95a800be70 comp rx=0 tx=0).stop 2023-07-06T02:24:05.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b3c62700 1 -- 172.21.15.89:0/4241901207 shutdown_connections 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b3c62700 1 --2- 172.21.15.89:0/4241901207 >> v2:172.21.15.172:3300/0 conn(0x7f95ac070e10 0x7f95ac071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b3c62700 1 --2- 172.21.15.89:0/4241901207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac15ce50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.302+0000 7f95b3c62700 1 -- 172.21.15.89:0/4241901207 >> 172.21.15.89:0/4241901207 conn(0x7f95ac0ce870 msgr2=0x7f95ac0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.303+0000 7f95b3c62700 1 -- 172.21.15.89:0/4241901207 shutdown_connections 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.303+0000 7f95b3c62700 1 -- 172.21.15.89:0/4241901207 wait complete. 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.304+0000 7f95b3c62700 1 Processor -- start 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.304+0000 7f95b3c62700 1 -- start start 2023-07-06T02:24:05.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b3c62700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac070e10 0x7f95ac14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b3c62700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95ac0ccea0 0x7f95ac0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b3c62700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b3c62700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0c8340 con 0x7f95ac158960 2023-07-06T02:24:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b3c62700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0b1f60 con 0x7f95ac070e10 2023-07-06T02:24:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b3c62700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac158f40 con 0x7f95ac0ccea0 2023-07-06T02:24:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac0c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34558/0 (socket says 172.21.15.89:34558) 2023-07-06T02:24:05.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 -- 172.21.15.89:0/1982292326 learned_addr learned my addr 172.21.15.89:0/1982292326 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:05.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b19fe700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac070e10 0x7f95ac14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b11fd700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95ac0ccea0 0x7f95ac0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 -- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95ac0ccea0 msgr2=0x7f95ac0c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95ac0ccea0 0x7f95ac0c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 -- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac070e10 msgr2=0x7f95ac14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac070e10 0x7f95ac14db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b21ff700 1 -- 172.21.15.89:0/1982292326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a800b7f0 con 0x7f95ac158960 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b19fe700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac070e10 0x7f95ac14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.305+0000 7f95b11fd700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95ac0ccea0 0x7f95ac0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.306+0000 7f95b21ff700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac0c4470 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007e40 tx=0x7f95a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.306+0000 7f95a27fc700 1 -- 172.21.15.89:0/1982292326 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a400eea0 con 0x7f95ac158960 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.306+0000 7f95a27fc700 1 -- 172.21.15.89:0/1982292326 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a4014420 con 0x7f95ac158960 2023-07-06T02:24:05.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.306+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0c49b0 con 0x7f95ac158960 2023-07-06T02:24:05.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.306+0000 7f95a27fc700 1 -- 172.21.15.89:0/1982292326 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40147b0 con 0x7f95ac158960 2023-07-06T02:24:05.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.306+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0c8060 con 0x7f95ac158960 2023-07-06T02:24:05.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.307+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac0b20a0 con 0x7f95ac158960 2023-07-06T02:24:05.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.307+0000 7f95a27fc700 1 -- 172.21.15.89:0/1982292326 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95a4014910 con 0x7f95ac158960 2023-07-06T02:24:05.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.307+0000 7f95a27fc700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f959006ace0 0x7f959006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.308+0000 7f95a27fc700 1 -- 172.21.15.89:0/1982292326 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f95a4089ae0 con 0x7f95ac158960 2023-07-06T02:24:05.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.308+0000 7f95b19fe700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f959006ace0 0x7f959006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.308+0000 7f95b19fe700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f959006ace0 0x7f959006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95a800b340 tx=0x7f95a8012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:05.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.312+0000 7f95a27fc700 1 -- 172.21.15.89:0/1982292326 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f95ac0b20a0 con 0x7f95ac158960 2023-07-06T02:24:05.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.577+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f95ac0b20a0 con 0x7f95ac158960 2023-07-06T02:24:05.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.583+0000 7f95a27fc700 1 -- 172.21.15.89:0/1982292326 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f95ac0b20a0 con 0x7f95ac158960 2023-07-06T02:24:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f959006ace0 msgr2=0x7f959006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f959006ace0 0x7f959006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95a800b340 tx=0x7f95a8012000 comp rx=0 tx=0).stop 2023-07-06T02:24:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 msgr2=0x7f95ac0c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac0c4470 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007e40 tx=0x7f95a400e3c0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 shutdown_connections 2023-07-06T02:24:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f959006ace0 0x7f959006d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95ac158960 0x7f95ac0c4470 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac070e10 0x7f95ac14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 --2- 172.21.15.89:0/1982292326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95ac0ccea0 0x7f95ac0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 >> 172.21.15.89:0/1982292326 conn(0x7f95ac0ce870 msgr2=0x7f95ac157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:05.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 shutdown_connections 2023-07-06T02:24:05.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.585+0000 7f95b3c62700 1 -- 172.21.15.89:0/1982292326 wait complete. 2023-07-06T02:24:05.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-06T02:24:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.718+0000 7fb65a30e700 1 Processor -- start 2023-07-06T02:24:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.718+0000 7fb65a30e700 1 -- start start 2023-07-06T02:24:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb65a30e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb654156ab0 0x7fb654160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb65a30e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6540cb3d0 con 0x7fb654160bb0 2023-07-06T02:24:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb65a30e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb654156e80 con 0x7fb6541600b0 2023-07-06T02:24:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb65a30e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6540ca370 con 0x7fb654156ab0 2023-07-06T02:24:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb6537fe700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb654160bb0 0x7fb6540b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59466/0 (socket says 172.21.15.89:59466) 2023-07-06T02:24:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb6537fe700 1 -- 172.21.15.89:0/3191610377 learned_addr learned my addr 172.21.15.89:0/3191610377 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822237945 0 0) 0x7fb6540cb3d0 con 0x7fb654160bb0 2023-07-06T02:24:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6400035b0 con 0x7fb654160bb0 2023-07-06T02:24:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb653fff700 1 --2- 172.21.15.89:0/3191610377 >> v2:172.21.15.172:3300/0 conn(0x7fb654156ab0 0x7fb654160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434511013 0 0) 0x7fb6400035b0 con 0x7fb654160bb0 2023-07-06T02:24:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 >> v2:172.21.15.172:3300/0 conn(0x7fb654156ab0 msgr2=0x7fb654160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 --2- 172.21.15.89:0/3191610377 >> v2:172.21.15.172:3300/0 conn(0x7fb654156ab0 0x7fb654160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 >> v1:172.21.15.172:6789/0 conn(0x7fb6541600b0 legacy=0x7fb6540cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540ca610 con 0x7fb654160bb0 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fb63c001670 con 0x7fb654160bb0 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 >> v1:172.21.15.89:6789/0 conn(0x7fb654160bb0 legacy=0x7fb6540b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.719+0000 7fb652ffd700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb640003a00 0x7fb640003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640004850 0x7fb640004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb640005200 0x7fb64000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540ca370 con 0x7fb640005200 2023-07-06T02:24:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64000c080 con 0x7fb640004850 2023-07-06T02:24:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64000c240 con 0x7fb640003a00 2023-07-06T02:24:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb640005200 0x7fb64000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6537fe700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640004850 0x7fb640004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb653fff700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb640003a00 0x7fb640003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 -- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb640003a00 msgr2=0x7fb640003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb640003a00 0x7fb640003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 -- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640004850 msgr2=0x7fb640004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640004850 0x7fb640004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 -- 172.21.15.89:0/3191610377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6400035b0 con 0x7fb640005200 2023-07-06T02:24:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb653fff700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb640003a00 0x7fb640003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 -- 172.21.15.89:0/3191610377 reap_dead start 2023-07-06T02:24:05.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb6588ab700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb640005200 0x7fb64000ba10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c190 tx=0x7fb640005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4701698c3f6c75e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c001db0 con 0x7fb640005200 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c00ccf0 con 0x7fb640005200 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.720+0000 7fb652ffd700 1 -- 172.21.15.89:0/3191610377 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c003c40 con 0x7fb640005200 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.721+0000 7fb65a30e700 1 -- 172.21.15.89:0/3191610377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb640005200 msgr2=0x7fb64000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.721+0000 7fb65a30e700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb640005200 0x7fb64000ba10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c190 tx=0x7fb640005880 comp rx=0 tx=0).stop 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.721+0000 7fb65a30e700 1 -- 172.21.15.89:0/3191610377 shutdown_connections 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.721+0000 7fb65a30e700 1 --2- 172.21.15.89:0/3191610377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb640005200 0x7fb64000ba10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.721+0000 7fb65a30e700 1 -- 172.21.15.89:0/3191610377 >> 172.21.15.89:0/3191610377 conn(0x7fb6540cc490 msgr2=0x7fb6540cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:05.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.721+0000 7fb65a30e700 1 -- 172.21.15.89:0/3191610377 shutdown_connections 2023-07-06T02:24:05.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.721+0000 7fb65a30e700 1 -- 172.21.15.89:0/3191610377 wait complete. 2023-07-06T02:24:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.722+0000 7fb65a30e700 1 Processor -- start 2023-07-06T02:24:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.722+0000 7fb65a30e700 1 -- start start 2023-07-06T02:24:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb65a30e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640005200 0x7fb6540b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb65a30e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6540b2da0 0x7fb6541448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb65a30e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb654144df0 0x7fb654155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb65a30e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541e7830 con 0x7fb654144df0 2023-07-06T02:24:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb65a30e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb654145270 con 0x7fb640005200 2023-07-06T02:24:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb65a30e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb654156170 con 0x7fb6540b2da0 2023-07-06T02:24:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb654144df0 0x7fb654155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb654144df0 0x7fb654155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34576/0 (socket says 172.21.15.89:34576) 2023-07-06T02:24:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 -- 172.21.15.89:0/2505753940 learned_addr learned my addr 172.21.15.89:0/2505753940 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 -- 172.21.15.89:0/2505753940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6540b2da0 msgr2=0x7fb6541448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6540b2da0 0x7fb6541448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 -- 172.21.15.89:0/2505753940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640005200 msgr2=0x7fb6540b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb653fff700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640005200 0x7fb6540b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640005200 0x7fb6540b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.723+0000 7fb6588ab700 1 -- 172.21.15.89:0/2505753940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6541e7a10 con 0x7fb654144df0 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.724+0000 7fb6588ab700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb654144df0 0x7fb654155230 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005040 tx=0x7fb64c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.724+0000 7fb650ff9700 1 -- 172.21.15.89:0/2505753940 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0052d0 con 0x7fb654144df0 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.724+0000 7fb650ff9700 1 -- 172.21.15.89:0/2505753940 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c005430 con 0x7fb654144df0 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.724+0000 7fb650ff9700 1 -- 172.21.15.89:0/2505753940 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0057a0 con 0x7fb654144df0 2023-07-06T02:24:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.724+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6541e7cf0 con 0x7fb654144df0 2023-07-06T02:24:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.724+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6541e82b0 con 0x7fb654144df0 2023-07-06T02:24:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.725+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb654074270 con 0x7fb654144df0 2023-07-06T02:24:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.725+0000 7fb650ff9700 1 -- 172.21.15.89:0/2505753940 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb64c01d480 con 0x7fb654144df0 2023-07-06T02:24:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.725+0000 7fb650ff9700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb63406ace0 0x7fb63406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.725+0000 7fb650ff9700 1 -- 172.21.15.89:0/2505753940 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb64c089ed0 con 0x7fb654144df0 2023-07-06T02:24:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.725+0000 7fb653fff700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb63406ace0 0x7fb63406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.726+0000 7fb653fff700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb63406ace0 0x7fb63406d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6440099b0 tx=0x7fb644008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:05.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.730+0000 7fb650ff9700 1 -- 172.21.15.89:0/2505753940 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb64c0552d0 con 0x7fb654144df0 2023-07-06T02:24:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.991+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb6540b5990 con 0x7fb654144df0 2023-07-06T02:24:05.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.995+0000 7fb650ff9700 1 -- 172.21.15.89:0/2505753940 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb64c0240a0 con 0x7fb654144df0 2023-07-06T02:24:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb63406ace0 msgr2=0x7fb63406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb63406ace0 0x7fb63406d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6440099b0 tx=0x7fb644008040 comp rx=0 tx=0).stop 2023-07-06T02:24:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb654144df0 msgr2=0x7fb654155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb654144df0 0x7fb654155230 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005040 tx=0x7fb64c0014a0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 shutdown_connections 2023-07-06T02:24:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fb63406ace0 0x7fb63406d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb654144df0 0x7fb654155230 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb640005200 0x7fb6540b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 --2- 172.21.15.89:0/2505753940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6540b2da0 0x7fb6541448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 >> 172.21.15.89:0/2505753940 conn(0x7fb6540cc490 msgr2=0x7fb6540b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 shutdown_connections 2023-07-06T02:24:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:05.997+0000 7fb65a30e700 1 -- 172.21.15.89:0/2505753940 wait complete. 2023-07-06T02:24:06.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-06T02:24:06.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.129+0000 7fa25fa15700 1 Processor -- start 2023-07-06T02:24:06.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.129+0000 7fa25fa15700 1 -- start start 2023-07-06T02:24:06.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25fa15700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa25815c450 0x7fa2581609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25fa15700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa258071b70 0x7fa25806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25fa15700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25806abb0 con 0x7fa258071700 2023-07-06T02:24:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25fa15700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2580ca380 con 0x7fa258071b70 2023-07-06T02:24:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25fa15700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580ca660 con 0x7fa25815c450 2023-07-06T02:24:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa258071b70 0x7fa25806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa258071b70 0x7fa25806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34586/0 (socket says 172.21.15.89:34586) 2023-07-06T02:24:06.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 -- 172.21.15.89:0/792485705 learned_addr learned my addr 172.21.15.89:0/792485705 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 -- 172.21.15.89:0/792485705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa25815c450 msgr2=0x7fa2581609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 --2- 172.21.15.89:0/792485705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa25815c450 0x7fa2581609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 -- 172.21.15.89:0/792485705 >> v1:172.21.15.172:6789/0 conn(0x7fa258071700 legacy=0x7fa25806d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 -- 172.21.15.89:0/792485705 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580ca940 con 0x7fa258071b70 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.130+0000 7fa25dfb2700 1 --2- 172.21.15.89:0/792485705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa258071b70 0x7fa25806e080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa250009e50 tx=0x7fa25000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23901182009b3d5b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa24ffff700 1 -- 172.21.15.89:0/792485705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25000e070 con 0x7fa258071b70 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa24ffff700 1 -- 172.21.15.89:0/792485705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250003a20 con 0x7fa258071b70 2023-07-06T02:24:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa24ffff700 1 -- 172.21.15.89:0/792485705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250003dd0 con 0x7fa258071b70 2023-07-06T02:24:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 -- 172.21.15.89:0/792485705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa258071b70 msgr2=0x7fa25806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 --2- 172.21.15.89:0/792485705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa258071b70 0x7fa25806e080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa250009e50 tx=0x7fa25000b330 comp rx=0 tx=0).stop 2023-07-06T02:24:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 -- 172.21.15.89:0/792485705 shutdown_connections 2023-07-06T02:24:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 --2- 172.21.15.89:0/792485705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa258071b70 0x7fa25806e080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 --2- 172.21.15.89:0/792485705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa25815c450 0x7fa2581609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 -- 172.21.15.89:0/792485705 >> 172.21.15.89:0/792485705 conn(0x7fa2580cf9c0 msgr2=0x7fa2580cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 -- 172.21.15.89:0/792485705 shutdown_connections 2023-07-06T02:24:06.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.131+0000 7fa25fa15700 1 -- 172.21.15.89:0/792485705 wait complete. 2023-07-06T02:24:06.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.132+0000 7fa25fa15700 1 Processor -- start 2023-07-06T02:24:06.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.132+0000 7fa25fa15700 1 -- start start 2023-07-06T02:24:06.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25fa15700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa258071700 0x7fa2581f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25fa15700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa258071b70 0x7fa25815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25fa15700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25815c450 0x7fa258155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25fa15700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581f3090 con 0x7fa25815c450 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25fa15700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25815aee0 con 0x7fa258071700 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25fa15700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25815b1c0 con 0x7fa258071b70 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25dfb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25815c450 0x7fa258155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25dfb2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25815c450 0x7fa258155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34592/0 (socket says 172.21.15.89:34592) 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25dfb2700 1 -- 172.21.15.89:0/3036140146 learned_addr learned my addr 172.21.15.89:0/3036140146 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25d7b1700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa258071700 0x7fa2581f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.133+0000 7fa25cfb0700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa258071b70 0x7fa25815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25dfb2700 1 -- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa258071b70 msgr2=0x7fa25815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25dfb2700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa258071b70 0x7fa25815a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25dfb2700 1 -- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa258071700 msgr2=0x7fa2581f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25dfb2700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa258071700 0x7fa2581f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25dfb2700 1 -- 172.21.15.89:0/3036140146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa250009b30 con 0x7fa25815c450 2023-07-06T02:24:06.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25d7b1700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa258071700 0x7fa2581f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:06.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25dfb2700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25815c450 0x7fa258155730 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa25806ebb0 tx=0x7fa250004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa24dffb700 1 -- 172.21.15.89:0/3036140146 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250004240 con 0x7fa25815c450 2023-07-06T02:24:06.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa24dffb700 1 -- 172.21.15.89:0/3036140146 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2500043a0 con 0x7fa25815c450 2023-07-06T02:24:06.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25815b4a0 con 0x7fa25815c450 2023-07-06T02:24:06.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa24dffb700 1 -- 172.21.15.89:0/3036140146 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250004750 con 0x7fa25815c450 2023-07-06T02:24:06.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.134+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25815ba60 con 0x7fa25815c450 2023-07-06T02:24:06.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.135+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa258074270 con 0x7fa25815c450 2023-07-06T02:24:06.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.135+0000 7fa24dffb700 1 -- 172.21.15.89:0/3036140146 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa250019070 con 0x7fa25815c450 2023-07-06T02:24:06.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.136+0000 7fa24dffb700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa23c06ad10 0x7fa23c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.136+0000 7fa24dffb700 1 -- 172.21.15.89:0/3036140146 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa25008a330 con 0x7fa25815c450 2023-07-06T02:24:06.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.136+0000 7fa25d7b1700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa23c06ad10 0x7fa23c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.136+0000 7fa25d7b1700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa23c06ad10 0x7fa23c06d1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa254006fd0 tx=0x7fa254008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.140+0000 7fa24dffb700 1 -- 172.21.15.89:0/3036140146 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa250055560 con 0x7fa25815c450 2023-07-06T02:24:06.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.401+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa2580cd830 con 0x7fa25815c450 2023-07-06T02:24:06.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.405+0000 7fa24dffb700 1 -- 172.21.15.89:0/3036140146 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa250058b80 con 0x7fa25815c450 2023-07-06T02:24:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa23c06ad10 msgr2=0x7fa23c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa23c06ad10 0x7fa23c06d1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa254006fd0 tx=0x7fa254008040 comp rx=0 tx=0).stop 2023-07-06T02:24:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25815c450 msgr2=0x7fa258155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25815c450 0x7fa258155730 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa25806ebb0 tx=0x7fa250004ad0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 shutdown_connections 2023-07-06T02:24:06.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa23c06ad10 0x7fa23c06d1c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25815c450 0x7fa258155730 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa258071700 0x7fa2581f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 --2- 172.21.15.89:0/3036140146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa258071b70 0x7fa25815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 >> 172.21.15.89:0/3036140146 conn(0x7fa2580cf9c0 msgr2=0x7fa2580b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 shutdown_connections 2023-07-06T02:24:06.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.407+0000 7fa25fa15700 1 -- 172.21.15.89:0/3036140146 wait complete. 2023-07-06T02:24:06.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.539+0000 7f6f107b7700 1 Processor -- start 2023-07-06T02:24:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.539+0000 7f6f107b7700 1 -- start start 2023-07-06T02:24:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f107b7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f0815c690 0x7f6f080cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f107b7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f080b1ba0 0x7f6f0806bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f107b7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f080cab50 con 0x7f6f080cbda0 2023-07-06T02:24:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f107b7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0815de50 con 0x7f6f0815c690 2023-07-06T02:24:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f107b7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0815db70 con 0x7f6f080b1ba0 2023-07-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f0e553700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f0815c690 0x7f6f080cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f0e553700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f0815c690 0x7f6f080cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34596/0 (socket says 172.21.15.89:34596) 2023-07-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.540+0000 7f6f0e553700 1 -- 172.21.15.89:0/117050814 learned_addr learned my addr 172.21.15.89:0/117050814 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0ed54700 1 --2- 172.21.15.89:0/117050814 >> v2:172.21.15.172:3300/0 conn(0x7f6f080b1ba0 0x7f6f0806bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0e553700 1 -- 172.21.15.89:0/117050814 >> v2:172.21.15.172:3300/0 conn(0x7f6f080b1ba0 msgr2=0x7f6f0806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0e553700 1 --2- 172.21.15.89:0/117050814 >> v2:172.21.15.172:3300/0 conn(0x7f6f080b1ba0 0x7f6f0806bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0e553700 1 -- 172.21.15.89:0/117050814 >> v1:172.21.15.172:6789/0 conn(0x7f6f080cbda0 legacy=0x7f6f080b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0e553700 1 -- 172.21.15.89:0/117050814 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f080cad80 con 0x7f6f0815c690 2023-07-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0e553700 1 --2- 172.21.15.89:0/117050814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f0815c690 0x7f6f080cb860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806bd20 tx=0x7f6ef800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f581987c48819774 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0d551700 1 -- 172.21.15.89:0/117050814 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800dcf0 con 0x7f6f0815c690 2023-07-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0d551700 1 -- 172.21.15.89:0/117050814 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef800de50 con 0x7f6f0815c690 2023-07-06T02:24:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f0d551700 1 -- 172.21.15.89:0/117050814 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8004250 con 0x7f6f0815c690 2023-07-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f107b7700 1 -- 172.21.15.89:0/117050814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f0815c690 msgr2=0x7f6f080cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.541+0000 7f6f107b7700 1 --2- 172.21.15.89:0/117050814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f0815c690 0x7f6f080cb860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806bd20 tx=0x7f6ef800be70 comp rx=0 tx=0).stop 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.542+0000 7f6f107b7700 1 -- 172.21.15.89:0/117050814 shutdown_connections 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.542+0000 7f6f107b7700 1 --2- 172.21.15.89:0/117050814 >> v2:172.21.15.172:3300/0 conn(0x7f6f080b1ba0 0x7f6f0806bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.542+0000 7f6f107b7700 1 --2- 172.21.15.89:0/117050814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f0815c690 0x7f6f080cb860 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.542+0000 7f6f107b7700 1 -- 172.21.15.89:0/117050814 >> 172.21.15.89:0/117050814 conn(0x7f6f080cc490 msgr2=0x7f6f080cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.542+0000 7f6f107b7700 1 -- 172.21.15.89:0/117050814 shutdown_connections 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.542+0000 7f6f107b7700 1 -- 172.21.15.89:0/117050814 wait complete. 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 Processor -- start 2023-07-06T02:24:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 -- start start 2023-07-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f080b1ba0 0x7f6f080c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f080cbda0 0x7f6f080c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f0815c690 0x7f6f080c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f080c4370 con 0x7f6f080cbda0 2023-07-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f080c6ae0 con 0x7f6f0815c690 2023-07-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.543+0000 7f6f107b7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f080c6dc0 con 0x7f6f080b1ba0 2023-07-06T02:24:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f080cbda0 0x7f6f080c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f080cbda0 0x7f6f080c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34604/0 (socket says 172.21.15.89:34604) 2023-07-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 -- 172.21.15.89:0/435491912 learned_addr learned my addr 172.21.15.89:0/435491912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0e553700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f080b1ba0 0x7f6f080c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0ed54700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f0815c690 0x7f6f080c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 -- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f080b1ba0 msgr2=0x7f6f080c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f080b1ba0 0x7f6f080c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 -- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f0815c690 msgr2=0x7f6f080c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f0815c690 0x7f6f080c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 -- 172.21.15.89:0/435491912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef800b7f0 con 0x7f6f080cbda0 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f0dd52700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f080cbda0 0x7f6f080c4790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c4a0 tx=0x7f6f0400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6efeffd700 1 -- 172.21.15.89:0/435491912 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04011790 con 0x7f6f080cbda0 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6efeffd700 1 -- 172.21.15.89:0/435491912 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f040118f0 con 0x7f6f080cbda0 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f080c70a0 con 0x7f6f080cbda0 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6efeffd700 1 -- 172.21.15.89:0/435491912 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04011ca0 con 0x7f6f080cbda0 2023-07-06T02:24:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.544+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f080c7660 con 0x7f6f080cbda0 2023-07-06T02:24:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.545+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f08074270 con 0x7f6f080cbda0 2023-07-06T02:24:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.546+0000 7f6efeffd700 1 -- 172.21.15.89:0/435491912 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f0401c460 con 0x7f6f080cbda0 2023-07-06T02:24:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.546+0000 7f6efeffd700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6eec06ace0 0x7f6eec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.546+0000 7f6efeffd700 1 -- 172.21.15.89:0/435491912 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6f04088e90 con 0x7f6f080cbda0 2023-07-06T02:24:06.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.546+0000 7f6f0e553700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6eec06ace0 0x7f6eec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.547+0000 7f6f0e553700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6eec06ace0 0x7f6eec06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000c00 tx=0x7f6ef801d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.550+0000 7f6efeffd700 1 -- 172.21.15.89:0/435491912 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6f040541e0 con 0x7f6f080cbda0 2023-07-06T02:24:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.737+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f080aec30 con 0x7f6f080cbda0 2023-07-06T02:24:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.738+0000 7f6efeffd700 1 -- 172.21.15.89:0/435491912 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+14774 (secure 0 0 0) 0x7f6f04057800 con 0x7f6f080cbda0 2023-07-06T02:24:06.741 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:06.741 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":34,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:24:03.499810+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:24:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6eec06ace0 msgr2=0x7f6eec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6eec06ace0 0x7f6eec06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000c00 tx=0x7f6ef801d040 comp rx=0 tx=0).stop 2023-07-06T02:24:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f080cbda0 msgr2=0x7f6f080c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f080cbda0 0x7f6f080c4790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c4a0 tx=0x7f6f0400c7b0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 shutdown_connections 2023-07-06T02:24:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f0815c690 0x7f6f080c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f080b1ba0 0x7f6f080c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f6eec06ace0 0x7f6eec06d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 --2- 172.21.15.89:0/435491912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f080cbda0 0x7f6f080c4790 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 >> 172.21.15.89:0/435491912 conn(0x7f6f080cc490 msgr2=0x7f6f0806b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 shutdown_connections 2023-07-06T02:24:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.740+0000 7f6f107b7700 1 -- 172.21.15.89:0/435491912 wait complete. 2023-07-06T02:24:06.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T02:24:06.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:24:06.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.876+0000 7fe8417e2700 1 Processor -- start 2023-07-06T02:24:06.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.876+0000 7fe8417e2700 1 -- start start 2023-07-06T02:24:06.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.876+0000 7fe8417e2700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe83c0ccfe0 0x7fe83c0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.876+0000 7fe8417e2700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c0b5960 con 0x7fe83c15c4f0 2023-07-06T02:24:06.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.876+0000 7fe8417e2700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c0cf4a0 con 0x7fe83c070920 2023-07-06T02:24:06.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.876+0000 7fe8417e2700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c0b38d0 con 0x7fe83c0ccfe0 2023-07-06T02:24:06.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe83c0ccfe0 0x7fe83c0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83affd700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe83c15c4f0 0x7fe83c070210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59512/0 (socket says 172.21.15.89:59512) 2023-07-06T02:24:06.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83a7fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe83c0ccfe0 0x7fe83c0b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34612/0 (socket says 172.21.15.89:34612) 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83affd700 1 -- 172.21.15.89:0/1422099172 learned_addr learned my addr 172.21.15.89:0/1422099172 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83a7fc700 1 -- 172.21.15.89:0/1422099172 >> v1:172.21.15.172:6789/0 conn(0x7fe83c070920 legacy=0x7fe83c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83a7fc700 1 -- 172.21.15.89:0/1422099172 >> v1:172.21.15.89:6789/0 conn(0x7fe83c15c4f0 legacy=0x7fe83c070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83a7fc700 1 -- 172.21.15.89:0/1422099172 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0b3bb0 con 0x7fe83c0ccfe0 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe839ffb700 1 -- 172.21.15.89:0/1422099172 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480389033 0 0) 0x7fe83c0b5960 con 0x7fe83c15c4f0 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe83a7fc700 1 --2- 172.21.15.89:0/1422099172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0b5420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009840 tx=0x7fe82c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4680b2a03cd957 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe839ffb700 1 -- 172.21.15.89:0/1422099172 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011830 con 0x7fe83c0ccfe0 2023-07-06T02:24:06.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe839ffb700 1 -- 172.21.15.89:0/1422099172 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c011990 con 0x7fe83c0ccfe0 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.877+0000 7fe839ffb700 1 -- 172.21.15.89:0/1422099172 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011d20 con 0x7fe83c0ccfe0 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.878+0000 7fe8417e2700 1 -- 172.21.15.89:0/1422099172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c0ccfe0 msgr2=0x7fe83c0b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.878+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1422099172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0b5420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009840 tx=0x7fe82c009b50 comp rx=0 tx=0).stop 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.878+0000 7fe8417e2700 1 -- 172.21.15.89:0/1422099172 shutdown_connections 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.878+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1422099172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0b5420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.878+0000 7fe8417e2700 1 -- 172.21.15.89:0/1422099172 >> 172.21.15.89:0/1422099172 conn(0x7fe83c0d09e0 msgr2=0x7fe83c0d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.878+0000 7fe8417e2700 1 -- 172.21.15.89:0/1422099172 shutdown_connections 2023-07-06T02:24:06.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.878+0000 7fe8417e2700 1 -- 172.21.15.89:0/1422099172 wait complete. 2023-07-06T02:24:06.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.879+0000 7fe8417e2700 1 Processor -- start 2023-07-06T02:24:06.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.879+0000 7fe8417e2700 1 -- start start 2023-07-06T02:24:06.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe8417e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c070920 0x7fe83c0c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe8417e2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe8417e2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe83c15c4f0 0x7fe83c0c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe8417e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c8ac0 con 0x7fe83c070920 2023-07-06T02:24:06.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe8417e2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c52e0 con 0x7fe83c0ccfe0 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe8417e2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c8890 con 0x7fe83c15c4f0 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe83affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c070920 0x7fe83c0c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.880+0000 7fe83affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c070920 0x7fe83c0c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34616/0 (socket says 172.21.15.89:34616) 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83affd700 1 -- 172.21.15.89:0/1017639823 learned_addr learned my addr 172.21.15.89:0/1017639823 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83a7fc700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83b7fe700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe83c15c4f0 0x7fe83c0c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83affd700 1 -- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe83c15c4f0 msgr2=0x7fe83c0c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83affd700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe83c15c4f0 0x7fe83c0c3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83affd700 1 -- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe83c0ccfe0 msgr2=0x7fe83c0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:06.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83affd700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:06.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83affd700 1 -- 172.21.15.89:0/1017639823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0094a0 con 0x7fe83c070920 2023-07-06T02:24:06.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83a7fc700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:06.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe83affd700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c070920 0x7fe83c0c7b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c520 tx=0x7fe83000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe8237fe700 1 -- 172.21.15.89:0/1017639823 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830004c80 con 0x7fe83c070920 2023-07-06T02:24:06.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe8237fe700 1 -- 172.21.15.89:0/1017639823 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830004e20 con 0x7fe83c070920 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c0c5e20 con 0x7fe83c070920 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe8237fe700 1 -- 172.21.15.89:0/1017639823 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830007310 con 0x7fe83c070920 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.881+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c0c54f0 con 0x7fe83c070920 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.882+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c0b41a0 con 0x7fe83c070920 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.883+0000 7fe8237fe700 1 -- 172.21.15.89:0/1017639823 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe830007470 con 0x7fe83c070920 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.883+0000 7fe8237fe700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe82806ace0 0x7fe82806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.883+0000 7fe8237fe700 1 -- 172.21.15.89:0/1017639823 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe830088fe0 con 0x7fe83c070920 2023-07-06T02:24:06.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.883+0000 7fe83a7fc700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe82806ace0 0x7fe82806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:06.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.883+0000 7fe83a7fc700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe82806ace0 0x7fe82806d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe83c06b3c0 tx=0x7fe82c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:06.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:06.887+0000 7fe8237fe700 1 -- 172.21.15.89:0/1017639823 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe830054210 con 0x7fe83c070920 2023-07-06T02:24:07.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.075+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe83c074270 con 0x7fe83c070920 2023-07-06T02:24:07.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.076+0000 7fe8237fe700 1 -- 172.21.15.89:0/1017639823 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe830057830 con 0x7fe83c070920 2023-07-06T02:24:07.079 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:24:07.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe82806ace0 msgr2=0x7fe82806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe82806ace0 0x7fe82806d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe83c06b3c0 tx=0x7fe82c01c040 comp rx=0 tx=0).stop 2023-07-06T02:24:07.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c070920 msgr2=0x7fe83c0c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c070920 0x7fe83c0c7b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c520 tx=0x7fe83000c8e0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 shutdown_connections 2023-07-06T02:24:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe83c15c4f0 0x7fe83c0c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe82806ace0 0x7fe82806d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe83c070920 0x7fe83c0c7b60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 --2- 172.21.15.89:0/1017639823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe83c0ccfe0 0x7fe83c0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 >> 172.21.15.89:0/1017639823 conn(0x7fe83c0d09e0 msgr2=0x7fe83c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 shutdown_connections 2023-07-06T02:24:07.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.078+0000 7fe8417e2700 1 -- 172.21.15.89:0/1017639823 wait complete. 2023-07-06T02:24:07.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:24:07.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.212+0000 7ff855e21700 1 Processor -- start 2023-07-06T02:24:07.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.212+0000 7ff855e21700 1 -- start start 2023-07-06T02:24:07.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff855e21700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff850156ab0 0x7ff850160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff855e21700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff850160bb0 0x7ff8500b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff855e21700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8500cb3d0 con 0x7ff8501600b0 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff855e21700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff850156e80 con 0x7ff850156ab0 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff855e21700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500ca370 con 0x7ff850160bb0 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff850156ab0 0x7ff850160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff850156ab0 0x7ff850160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34624/0 (socket says 172.21.15.89:34624) 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84f7fe700 1 -- 172.21.15.89:0/3041342127 learned_addr learned my addr 172.21.15.89:0/3041342127 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84effd700 1 --2- 172.21.15.89:0/3041342127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff850160bb0 0x7ff8500b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84f7fe700 1 -- 172.21.15.89:0/3041342127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff850160bb0 msgr2=0x7ff8500b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84f7fe700 1 --2- 172.21.15.89:0/3041342127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff850160bb0 0x7ff8500b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84f7fe700 1 -- 172.21.15.89:0/3041342127 >> v1:172.21.15.172:6789/0 conn(0x7ff8501600b0 legacy=0x7ff85006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.213+0000 7ff84f7fe700 1 -- 172.21.15.89:0/3041342127 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500ca610 con 0x7ff850156ab0 2023-07-06T02:24:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff84f7fe700 1 --2- 172.21.15.89:0/3041342127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff850156ab0 0x7ff850160670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff8380098f0 tx=0x7ff838009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d80a7155cb2b9d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff84e7fc700 1 -- 172.21.15.89:0/3041342127 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83800b8a0 con 0x7ff850156ab0 2023-07-06T02:24:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff84e7fc700 1 -- 172.21.15.89:0/3041342127 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83800ba00 con 0x7ff850156ab0 2023-07-06T02:24:07.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff84e7fc700 1 -- 172.21.15.89:0/3041342127 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83800bd90 con 0x7ff850156ab0 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 -- 172.21.15.89:0/3041342127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff850156ab0 msgr2=0x7ff850160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 --2- 172.21.15.89:0/3041342127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff850156ab0 0x7ff850160670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff8380098f0 tx=0x7ff838009c00 comp rx=0 tx=0).stop 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 -- 172.21.15.89:0/3041342127 shutdown_connections 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 --2- 172.21.15.89:0/3041342127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff850156ab0 0x7ff850160670 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 --2- 172.21.15.89:0/3041342127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff850160bb0 0x7ff8500b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 -- 172.21.15.89:0/3041342127 >> 172.21.15.89:0/3041342127 conn(0x7ff8500cc490 msgr2=0x7ff8500cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 -- 172.21.15.89:0/3041342127 shutdown_connections 2023-07-06T02:24:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.214+0000 7ff855e21700 1 -- 172.21.15.89:0/3041342127 wait complete. 2023-07-06T02:24:07.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.215+0000 7ff855e21700 1 Processor -- start 2023-07-06T02:24:07.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.216+0000 7ff855e21700 1 -- start start 2023-07-06T02:24:07.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.216+0000 7ff855e21700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850156ab0 0x7ff850142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.216+0000 7ff855e21700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8501600b0 0x7ff8501e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.216+0000 7ff855e21700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8500b2ec0 0x7ff8500b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.216+0000 7ff855e21700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b45a0 con 0x7ff8501600b0 2023-07-06T02:24:07.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.216+0000 7ff855e21700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b3920 con 0x7ff850156ab0 2023-07-06T02:24:07.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.216+0000 7ff855e21700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b3c00 con 0x7ff8500b2ec0 2023-07-06T02:24:07.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8501600b0 0x7ff8501e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8501600b0 0x7ff8501e5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34630/0 (socket says 172.21.15.89:34630) 2023-07-06T02:24:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 -- 172.21.15.89:0/2718611267 learned_addr learned my addr 172.21.15.89:0/2718611267 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84f7fe700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850156ab0 0x7ff850142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84ffff700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8500b2ec0 0x7ff8500b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 -- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8500b2ec0 msgr2=0x7ff8500b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8500b2ec0 0x7ff8500b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 -- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850156ab0 msgr2=0x7ff850142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850156ab0 0x7ff850142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 -- 172.21.15.89:0/2718611267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff838009550 con 0x7ff8501600b0 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff84effd700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8501600b0 0x7ff8501e5ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff84000d800 tx=0x7ff84000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff847fff700 1 -- 172.21.15.89:0/2718611267 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400125f0 con 0x7ff8501600b0 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff847fff700 1 -- 172.21.15.89:0/2718611267 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff840012750 con 0x7ff8501600b0 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8500b3ee0 con 0x7ff8501600b0 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff847fff700 1 -- 172.21.15.89:0/2718611267 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840012b00 con 0x7ff8501600b0 2023-07-06T02:24:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.217+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff850155f20 con 0x7ff8501600b0 2023-07-06T02:24:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.218+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff850074270 con 0x7ff8501600b0 2023-07-06T02:24:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.219+0000 7ff847fff700 1 -- 172.21.15.89:0/2718611267 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff840012c60 con 0x7ff8501600b0 2023-07-06T02:24:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.219+0000 7ff847fff700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff83006ace0 0x7ff83006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.219+0000 7ff847fff700 1 -- 172.21.15.89:0/2718611267 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff840089270 con 0x7ff8501600b0 2023-07-06T02:24:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.219+0000 7ff84f7fe700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff83006ace0 0x7ff83006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.219+0000 7ff84f7fe700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff83006ace0 0x7ff83006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff838009c40 tx=0x7ff838008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.223+0000 7ff847fff700 1 -- 172.21.15.89:0/2718611267 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff840054670 con 0x7ff8501600b0 2023-07-06T02:24:07.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.410+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8500b5990 con 0x7ff8501600b0 2023-07-06T02:24:07.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.411+0000 7ff847fff700 1 -- 172.21.15.89:0/2718611267 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff840057c90 con 0x7ff8501600b0 2023-07-06T02:24:07.414 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:24:07.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.412+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff83006ace0 msgr2=0x7ff83006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.412+0000 7ff855e21700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff83006ace0 0x7ff83006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff838009c40 tx=0x7ff838008030 comp rx=0 tx=0).stop 2023-07-06T02:24:07.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8501600b0 msgr2=0x7ff8501e5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8501600b0 0x7ff8501e5ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff84000d800 tx=0x7ff84000db10 comp rx=0 tx=0).stop 2023-07-06T02:24:07.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 shutdown_connections 2023-07-06T02:24:07.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8500b2ec0 0x7ff8500b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850156ab0 0x7ff850142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff83006ace0 0x7ff83006d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 --2- 172.21.15.89:0/2718611267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8501600b0 0x7ff8501e5ff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 >> 172.21.15.89:0/2718611267 conn(0x7ff8500cc490 msgr2=0x7ff8500cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:07.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 shutdown_connections 2023-07-06T02:24:07.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.413+0000 7ff855e21700 1 -- 172.21.15.89:0/2718611267 wait complete. 2023-07-06T02:24:07.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T02:24:07.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.547+0000 7f5d72a72700 1 Processor -- start 2023-07-06T02:24:07.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.547+0000 7f5d72a72700 1 -- start start 2023-07-06T02:24:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d72a72700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d6c06dc10 0x7f5d6c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d72a72700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6c0cd350 con 0x7f5d6c15adb0 2023-07-06T02:24:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d72a72700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6c06e110 con 0x7f5d6c0705a0 2023-07-06T02:24:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d72a72700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c06e3f0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d6c06dc10 0x7f5d6c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d7100f700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5d6c15adb0 0x7f5d6c157950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59542/0 (socket says 172.21.15.89:59542) 2023-07-06T02:24:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6bfff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d6c06dc10 0x7f5d6c15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34638/0 (socket says 172.21.15.89:34638) 2023-07-06T02:24:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d7100f700 1 -- 172.21.15.89:0/3935971379 learned_addr learned my addr 172.21.15.89:0/3935971379 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6bfff700 1 -- 172.21.15.89:0/3935971379 >> v1:172.21.15.172:6789/0 conn(0x7f5d6c0705a0 legacy=0x7f5d6c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6bfff700 1 -- 172.21.15.89:0/3935971379 >> v1:172.21.15.89:6789/0 conn(0x7f5d6c15adb0 legacy=0x7f5d6c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6b7fe700 1 -- 172.21.15.89:0/3935971379 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049944116 0 0) 0x7f5d6c0cd350 con 0x7f5d6c15adb0 2023-07-06T02:24:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6bfff700 1 -- 172.21.15.89:0/3935971379 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c06e6d0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6bfff700 1 --2- 172.21.15.89:0/3935971379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c15a870 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009870 tx=0x7f5d58009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed66ff56547c138 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6b7fe700 1 -- 172.21.15.89:0/3935971379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58004230 con 0x7f5d6c06dc10 2023-07-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6b7fe700 1 -- 172.21.15.89:0/3935971379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58004390 con 0x7f5d6c06dc10 2023-07-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.548+0000 7f5d6b7fe700 1 -- 172.21.15.89:0/3935971379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580036a0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.549+0000 7f5d72a72700 1 -- 172.21.15.89:0/3935971379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 msgr2=0x7f5d6c15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.549+0000 7f5d72a72700 1 --2- 172.21.15.89:0/3935971379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c15a870 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009870 tx=0x7f5d58009b80 comp rx=0 tx=0).stop 2023-07-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.549+0000 7f5d72a72700 1 -- 172.21.15.89:0/3935971379 shutdown_connections 2023-07-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.549+0000 7f5d72a72700 1 --2- 172.21.15.89:0/3935971379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c15a870 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.549+0000 7f5d72a72700 1 -- 172.21.15.89:0/3935971379 >> 172.21.15.89:0/3935971379 conn(0x7f5d6c0d0b00 msgr2=0x7f5d6c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:07.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.549+0000 7f5d72a72700 1 -- 172.21.15.89:0/3935971379 shutdown_connections 2023-07-06T02:24:07.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.549+0000 7f5d72a72700 1 -- 172.21.15.89:0/3935971379 wait complete. 2023-07-06T02:24:07.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.550+0000 7f5d72a72700 1 Processor -- start 2023-07-06T02:24:07.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.550+0000 7f5d72a72700 1 -- start start 2023-07-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d72a72700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d72a72700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d6c0705a0 0x7f5d6c0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d72a72700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d6c15adb0 0x7f5d6c0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d72a72700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c1485a0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d72a72700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0c9aa0 con 0x7f5d6c15adb0 2023-07-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d72a72700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c1482c0 con 0x7f5d6c0705a0 2023-07-06T02:24:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c144e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34642/0 (socket says 172.21.15.89:34642) 2023-07-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 -- 172.21.15.89:0/1386510809 learned_addr learned my addr 172.21.15.89:0/1386510809 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7100f700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d6c15adb0 0x7f5d6c0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d6bfff700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d6c0705a0 0x7f5d6c0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 -- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d6c0705a0 msgr2=0x7f5d6c0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d6c0705a0 0x7f5d6c0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 -- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d6c15adb0 msgr2=0x7f5d6c0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d6c15adb0 0x7f5d6c0b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 -- 172.21.15.89:0/1386510809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d580094a0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d6bfff700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d6c0705a0 0x7f5d6c0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7100f700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d6c15adb0 0x7f5d6c0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.551+0000 7f5d7080e700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c144e50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c4d0 tx=0x7f5d6000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.552+0000 7f5d697fa700 1 -- 172.21.15.89:0/1386510809 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60011790 con 0x7f5d6c06dc10 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.552+0000 7f5d697fa700 1 -- 172.21.15.89:0/1386510809 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d600118f0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.552+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c0c9d40 con 0x7f5d6c06dc10 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.552+0000 7f5d697fa700 1 -- 172.21.15.89:0/1386510809 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60011ca0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.552+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c0ca300 con 0x7f5d6c06dc10 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.553+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c074270 con 0x7f5d6c06dc10 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.553+0000 7f5d697fa700 1 -- 172.21.15.89:0/1386510809 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d6001c460 con 0x7f5d6c06dc10 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.553+0000 7f5d697fa700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d5006ace0 0x7f5d5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.554+0000 7f5d697fa700 1 -- 172.21.15.89:0/1386510809 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5d600890c0 con 0x7f5d6c06dc10 2023-07-06T02:24:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.554+0000 7f5d6bfff700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d5006ace0 0x7f5d5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.554+0000 7f5d6bfff700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d5006ace0 0x7f5d5006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097c0 tx=0x7f5d5801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.558+0000 7f5d697fa700 1 -- 172.21.15.89:0/1386510809 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d60054370 con 0x7f5d6c06dc10 2023-07-06T02:24:07.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.747+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5d6c06f590 con 0x7f5d6c06dc10 2023-07-06T02:24:07.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.747+0000 7f5d697fa700 1 -- 172.21.15.89:0/1386510809 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5d60057990 con 0x7f5d6c06dc10 2023-07-06T02:24:07.750 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:07.751 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-07-06T02:24:07.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:07.751 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-07-06T02:24:07.751 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": "rbd" 2023-07-06T02:24:07.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-07-06T02:24:07.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-07-06T02:24:07.752 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 2, 2023-07-06T02:24:07.753 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-07-06T02:24:07.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-07-06T02:24:07.753 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-07-06T02:24:07.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d5006ace0 msgr2=0x7f5d5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d5006ace0 0x7f5d5006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097c0 tx=0x7f5d5801c040 comp rx=0 tx=0).stop 2023-07-06T02:24:07.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 msgr2=0x7f5d6c144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c144e50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c4d0 tx=0x7f5d6000c890 comp rx=0 tx=0).stop 2023-07-06T02:24:07.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 shutdown_connections 2023-07-06T02:24:07.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d6c15adb0 0x7f5d6c0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f5d5006ace0 0x7f5d5006d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d6c06dc10 0x7f5d6c144e50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 --2- 172.21.15.89:0/1386510809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d6c0705a0 0x7f5d6c0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 >> 172.21.15.89:0/1386510809 conn(0x7f5d6c0d0b00 msgr2=0x7f5d6c070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:07.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 shutdown_connections 2023-07-06T02:24:07.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.749+0000 7f5d72a72700 1 -- 172.21.15.89:0/1386510809 wait complete. 2023-07-06T02:24:07.764 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T02:24:07.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T02:24:07.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.885+0000 7f3320c88700 1 Processor -- start 2023-07-06T02:24:07.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.885+0000 7f3320c88700 1 -- start start 2023-07-06T02:24:07.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f3320c88700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f3320c88700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f331c0cd580 0x7f331c071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f3320c88700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f331c071bb0 0x7f331c0ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f3320c88700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c072030 con 0x7f331c071bb0 2023-07-06T02:24:07.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f3320c88700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c15ce40 con 0x7f331c0cd580 2023-07-06T02:24:07.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f3320c88700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c15d120 con 0x7f331c15a160 2023-07-06T02:24:07.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f331c071bb0 0x7f331c0ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f331c071bb0 0x7f331c0ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34652/0 (socket says 172.21.15.89:34652) 2023-07-06T02:24:07.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 -- 172.21.15.89:0/1638753648 learned_addr learned my addr 172.21.15.89:0/1638753648 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:07.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f3319d9b700 1 --2- 172.21.15.89:0/1638753648 >> v2:172.21.15.172:3300/0 conn(0x7f331c0cd580 0x7f331c071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331a59c700 1 --2- 172.21.15.89:0/1638753648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 -- 172.21.15.89:0/1638753648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 msgr2=0x7f331c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:07.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 --2- 172.21.15.89:0/1638753648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 -- 172.21.15.89:0/1638753648 >> v2:172.21.15.172:3300/0 conn(0x7f331c0cd580 msgr2=0x7f331c071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:07.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 --2- 172.21.15.89:0/1638753648 >> v2:172.21.15.172:3300/0 conn(0x7f331c0cd580 0x7f331c071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.886+0000 7f331ad9d700 1 -- 172.21.15.89:0/1638753648 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c15d400 con 0x7f331c071bb0 2023-07-06T02:24:07.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f331ad9d700 1 --2- 172.21.15.89:0/1638753648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c071bb0 0x7f331c0ca3a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f331c06f3e0 tx=0x7f331400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f982fd9627c8035 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f331959a700 1 -- 172.21.15.89:0/1638753648 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400e070 con 0x7f331c071bb0 2023-07-06T02:24:07.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f331959a700 1 -- 172.21.15.89:0/1638753648 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331400b9c0 con 0x7f331c071bb0 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f331959a700 1 -- 172.21.15.89:0/1638753648 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400bd70 con 0x7f331c071bb0 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 -- 172.21.15.89:0/1638753648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c071bb0 msgr2=0x7f331c0ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 --2- 172.21.15.89:0/1638753648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c071bb0 0x7f331c0ca3a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f331c06f3e0 tx=0x7f331400b330 comp rx=0 tx=0).stop 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 -- 172.21.15.89:0/1638753648 shutdown_connections 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 --2- 172.21.15.89:0/1638753648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 --2- 172.21.15.89:0/1638753648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c071bb0 0x7f331c0ca3a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 --2- 172.21.15.89:0/1638753648 >> v2:172.21.15.172:3300/0 conn(0x7f331c0cd580 0x7f331c071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 -- 172.21.15.89:0/1638753648 >> 172.21.15.89:0/1638753648 conn(0x7f331c0cd080 msgr2=0x7f331c0cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:07.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 -- 172.21.15.89:0/1638753648 shutdown_connections 2023-07-06T02:24:07.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.887+0000 7f3320c88700 1 -- 172.21.15.89:0/1638753648 wait complete. 2023-07-06T02:24:07.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 Processor -- start 2023-07-06T02:24:07.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 -- start start 2023-07-06T02:24:07.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f331c071bb0 0x7f331c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c0cd580 0x7f331c0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0c8470 con 0x7f331c0cd580 2023-07-06T02:24:07.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0c8750 con 0x7f331c071bb0 2023-07-06T02:24:07.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.889+0000 7f3320c88700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0c8a30 con 0x7f331c15a160 2023-07-06T02:24:07.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c0cd580 0x7f331c0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c0cd580 0x7f331c0c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34656/0 (socket says 172.21.15.89:34656) 2023-07-06T02:24:07.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 -- 172.21.15.89:0/1469126692 learned_addr learned my addr 172.21.15.89:0/1469126692 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f331a59c700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f331c071bb0 0x7f331c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f331ad9d700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 -- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 msgr2=0x7f331c15e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c15e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 -- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f331c071bb0 msgr2=0x7f331c0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f331c071bb0 0x7f331c0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 -- 172.21.15.89:0/1469126692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314009b30 con 0x7f331c0cd580 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3319d9b700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c0cd580 0x7f331c0c7b70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f330c00e810 tx=0x7f330c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f330affd700 1 -- 172.21.15.89:0/1469126692 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c00c070 con 0x7f331c0cd580 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f330affd700 1 -- 172.21.15.89:0/1469126692 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c013610 con 0x7f331c0cd580 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c0c8d10 con 0x7f331c0cd580 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f330affd700 1 -- 172.21.15.89:0/1469126692 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c0139a0 con 0x7f331c0cd580 2023-07-06T02:24:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.890+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c0c80b0 con 0x7f331c0cd580 2023-07-06T02:24:07.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.891+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c0c9230 con 0x7f331c0cd580 2023-07-06T02:24:07.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.891+0000 7f330affd700 1 -- 172.21.15.89:0/1469126692 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f330c013b00 con 0x7f331c0cd580 2023-07-06T02:24:07.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.892+0000 7f330affd700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f32fc06ace0 0x7f32fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:07.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.892+0000 7f330affd700 1 -- 172.21.15.89:0/1469126692 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f330c089090 con 0x7f331c0cd580 2023-07-06T02:24:07.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.892+0000 7f331a59c700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f32fc06ace0 0x7f32fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:07.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.892+0000 7f331a59c700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f32fc06ace0 0x7f32fc06d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3310006b20 tx=0x7f3310008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:07.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:07.896+0000 7f330affd700 1 -- 172.21.15.89:0/1469126692 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f331c0c9230 con 0x7f331c0cd580 2023-07-06T02:24:08.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.084+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f331c0c9230 con 0x7f331c0cd580 2023-07-06T02:24:08.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.506+0000 7f330affd700 1 -- 172.21.15.89:0/1469126692 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f331c0c9230 con 0x7f331c0cd580 2023-07-06T02:24:08.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.510+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f32fc06ace0 msgr2=0x7f32fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:08.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f32fc06ace0 0x7f32fc06d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3310006b20 tx=0x7f3310008040 comp rx=0 tx=0).stop 2023-07-06T02:24:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c0cd580 msgr2=0x7f331c0c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c0cd580 0x7f331c0c7b70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f330c00e810 tx=0x7f330c00eb20 comp rx=0 tx=0).stop 2023-07-06T02:24:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 shutdown_connections 2023-07-06T02:24:08.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331c15a160 0x7f331c15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f331c071bb0 0x7f331c0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f32fc06ace0 0x7f32fc06d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 --2- 172.21.15.89:0/1469126692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f331c0cd580 0x7f331c0c7b70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 >> 172.21.15.89:0/1469126692 conn(0x7f331c0cd080 msgr2=0x7f331c0ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 shutdown_connections 2023-07-06T02:24:08.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.511+0000 7f3320c88700 1 -- 172.21.15.89:0/1469126692 wait complete. 2023-07-06T02:24:08.516 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-07-06T02:24:08.528 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T02:24:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.655+0000 7f15b1df6700 1 Processor -- start 2023-07-06T02:24:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.655+0000 7f15b1df6700 1 -- start start 2023-07-06T02:24:08.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15b1df6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac156ab0 0x7f15ac160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:08.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15b1df6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15ac160bb0 0x7f15ac0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15b1df6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac0cb3d0 con 0x7f15ac1600b0 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15b1df6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac156e80 con 0x7f15ac160bb0 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15b1df6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0ca370 con 0x7f15ac156ab0 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15a2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15ac160bb0 0x7f15ac0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15a2ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15ac160bb0 0x7f15ac0b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34664/0 (socket says 172.21.15.89:34664) 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15a2ffd700 1 -- 172.21.15.89:0/1309542675 learned_addr learned my addr 172.21.15.89:0/1309542675 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15ab7fe700 1 --2- 172.21.15.89:0/1309542675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac156ab0 0x7f15ac160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15a2ffd700 1 -- 172.21.15.89:0/1309542675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac156ab0 msgr2=0x7f15ac160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15a2ffd700 1 --2- 172.21.15.89:0/1309542675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac156ab0 0x7f15ac160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15a2ffd700 1 -- 172.21.15.89:0/1309542675 >> v1:172.21.15.172:6789/0 conn(0x7f15ac1600b0 legacy=0x7f15ac06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15a2ffd700 1 -- 172.21.15.89:0/1309542675 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0ca610 con 0x7f15ac160bb0 2023-07-06T02:24:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.656+0000 7f15ab7fe700 1 --2- 172.21.15.89:0/1309542675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac156ab0 0x7f15ac160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15a2ffd700 1 --2- 172.21.15.89:0/1309542675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac160bb0 0x7f15ac0b1490 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f15940098f0 tx=0x7f1594009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57583f4f6813abb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15aaffd700 1 -- 172.21.15.89:0/1309542675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594004230 con 0x7f15ac160bb0 2023-07-06T02:24:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15aaffd700 1 -- 172.21.15.89:0/1309542675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594004390 con 0x7f15ac160bb0 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15aaffd700 1 -- 172.21.15.89:0/1309542675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15940036a0 con 0x7f15ac160bb0 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 -- 172.21.15.89:0/1309542675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac160bb0 msgr2=0x7f15ac0b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 --2- 172.21.15.89:0/1309542675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac160bb0 0x7f15ac0b1490 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f15940098f0 tx=0x7f1594009c00 comp rx=0 tx=0).stop 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 -- 172.21.15.89:0/1309542675 shutdown_connections 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 --2- 172.21.15.89:0/1309542675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac156ab0 0x7f15ac160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 --2- 172.21.15.89:0/1309542675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac160bb0 0x7f15ac0b1490 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 -- 172.21.15.89:0/1309542675 >> 172.21.15.89:0/1309542675 conn(0x7f15ac0cc490 msgr2=0x7f15ac0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 -- 172.21.15.89:0/1309542675 shutdown_connections 2023-07-06T02:24:08.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.657+0000 7f15b1df6700 1 -- 172.21.15.89:0/1309542675 wait complete. 2023-07-06T02:24:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.658+0000 7f15b1df6700 1 Processor -- start 2023-07-06T02:24:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15b1df6700 1 -- start start 2023-07-06T02:24:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15b1df6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15ac156ab0 0x7f15ac155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15b1df6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac1600b0 0x7f15ac0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15b1df6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac0b2da0 0x7f15ac0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15b1df6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac156450 con 0x7f15ac0b2da0 2023-07-06T02:24:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15b1df6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1f6070 con 0x7f15ac156ab0 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15b1df6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0ad3d0 con 0x7f15ac1600b0 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.659+0000 7f15abfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac0b2da0 0x7f15ac0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac0b2da0 0x7f15ac0ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34670/0 (socket says 172.21.15.89:34670) 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 -- 172.21.15.89:0/725610568 learned_addr learned my addr 172.21.15.89:0/725610568 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15ab7fe700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15ac156ab0 0x7f15ac155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15a2ffd700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac1600b0 0x7f15ac0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 -- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac1600b0 msgr2=0x7f15ac0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac1600b0 0x7f15ac0b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 -- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15ac156ab0 msgr2=0x7f15ac155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15ac156ab0 0x7f15ac155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 -- 172.21.15.89:0/725610568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594009550 con 0x7f15ac0b2da0 2023-07-06T02:24:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15abfff700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac0b2da0 0x7f15ac0ae190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f15a400cba0 tx=0x7f15a400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15a8ff9700 1 -- 172.21.15.89:0/725610568 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a400aaa0 con 0x7f15ac0b2da0 2023-07-06T02:24:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15a8ff9700 1 -- 172.21.15.89:0/725610568 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a400ac00 con 0x7f15ac0b2da0 2023-07-06T02:24:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac0ad6b0 con 0x7f15ac0b2da0 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15a8ff9700 1 -- 172.21.15.89:0/725610568 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4012420 con 0x7f15ac0b2da0 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.660+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac0adc70 con 0x7f15ac0b2da0 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.661+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac074270 con 0x7f15ac0b2da0 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.662+0000 7f15a8ff9700 1 -- 172.21.15.89:0/725610568 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15a4012580 con 0x7f15ac0b2da0 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.662+0000 7f15a8ff9700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f159006ace0 0x7f159006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.662+0000 7f15a8ff9700 1 -- 172.21.15.89:0/725610568 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f15a4089a50 con 0x7f15ac0b2da0 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.662+0000 7f15ab7fe700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f159006ace0 0x7f159006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.662+0000 7f15ab7fe700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f159006ace0 0x7f159006d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:08.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.667+0000 7f15a8ff9700 1 -- 172.21.15.89:0/725610568 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f15a4054ba0 con 0x7f15ac0b2da0 2023-07-06T02:24:08.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:08.853+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f15ac0b5990 con 0x7f15ac0b2da0 2023-07-06T02:24:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.510+0000 7f15a8ff9700 1 -- 172.21.15.89:0/725610568 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7f15a4016020 con 0x7f15ac0b2da0 2023-07-06T02:24:09.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.515+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f159006ace0 msgr2=0x7f159006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.515+0000 7f15b1df6700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f159006ace0 0x7f159006d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).stop 2023-07-06T02:24:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.515+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac0b2da0 msgr2=0x7f15ac0ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:09.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.516+0000 7f15b1df6700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac0b2da0 0x7f15ac0ae190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f15a400cba0 tx=0x7f15a400ceb0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.519+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 shutdown_connections 2023-07-06T02:24:09.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.519+0000 7f15b1df6700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f159006ace0 0x7f159006d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.519+0000 7f15b1df6700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15ac0b2da0 0x7f15ac0ae190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.519+0000 7f15b1df6700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15ac156ab0 0x7f15ac155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.519+0000 7f15b1df6700 1 --2- 172.21.15.89:0/725610568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15ac1600b0 0x7f15ac0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.519+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 >> 172.21.15.89:0/725610568 conn(0x7f15ac0cc490 msgr2=0x7f15ac0cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:09.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.521+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 shutdown_connections 2023-07-06T02:24:09.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.521+0000 7f15b1df6700 1 -- 172.21.15.89:0/725610568 wait complete. 2023-07-06T02:24:09.527 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-07-06T02:24:09.542 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T02:24:09.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.664+0000 7f04cd71e700 1 Processor -- start 2023-07-06T02:24:09.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.664+0000 7f04cd71e700 1 -- start start 2023-07-06T02:24:09.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.664+0000 7f04cd71e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04c80ccfe0 0x7f04c80b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:09.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.664+0000 7f04cd71e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f04c8070920 0x7f04c80cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:09.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.664+0000 7f04cd71e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c80b5960 con 0x7f04c815c4f0 2023-07-06T02:24:09.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.664+0000 7f04cd71e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c80cf4a0 con 0x7f04c80ccfe0 2023-07-06T02:24:09.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04cd71e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c80b38d0 con 0x7f04c8070920 2023-07-06T02:24:09.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04c80ccfe0 0x7f04c80b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:09.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04c80ccfe0 0x7f04c80b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34678/0 (socket says 172.21.15.89:34678) 2023-07-06T02:24:09.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 -- 172.21.15.89:0/4184590649 learned_addr learned my addr 172.21.15.89:0/4184590649 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:09.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c6ffd700 1 --1- 172.21.15.89:0/4184590649 >> v1:172.21.15.89:6789/0 conn(0x7f04c815c4f0 0x7f04c8070210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59578/0 (socket says 172.21.15.89:59578) 2023-07-06T02:24:09.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 -- 172.21.15.89:0/4184590649 >> v2:172.21.15.172:3300/0 conn(0x7f04c8070920 msgr2=0x7f04c80cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:09.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 --2- 172.21.15.89:0/4184590649 >> v2:172.21.15.172:3300/0 conn(0x7f04c8070920 0x7f04c80cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 -- 172.21.15.89:0/4184590649 >> v1:172.21.15.89:6789/0 conn(0x7f04c815c4f0 legacy=0x7f04c8070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 -- 172.21.15.89:0/4184590649 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c80b3bb0 con 0x7f04c80ccfe0 2023-07-06T02:24:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c5ffb700 1 -- 172.21.15.89:0/4184590649 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157695474 0 0) 0x7f04c80b5960 con 0x7f04c815c4f0 2023-07-06T02:24:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c67fc700 1 --2- 172.21.15.89:0/4184590649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80b5420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f04c80c7a40 tx=0x7f04bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de166dc42caabc99 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c5ffb700 1 -- 172.21.15.89:0/4184590649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00b870 con 0x7f04c80ccfe0 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c5ffb700 1 -- 172.21.15.89:0/4184590649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc00b9d0 con 0x7f04c80ccfe0 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.665+0000 7f04c5ffb700 1 -- 172.21.15.89:0/4184590649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00bd60 con 0x7f04c80ccfe0 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 -- 172.21.15.89:0/4184590649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 msgr2=0x7f04c80b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 --2- 172.21.15.89:0/4184590649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80b5420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f04c80c7a40 tx=0x7f04bc009bd0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 -- 172.21.15.89:0/4184590649 shutdown_connections 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 --2- 172.21.15.89:0/4184590649 >> v2:172.21.15.172:3300/0 conn(0x7f04c8070920 0x7f04c80cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 --2- 172.21.15.89:0/4184590649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80b5420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 -- 172.21.15.89:0/4184590649 >> 172.21.15.89:0/4184590649 conn(0x7f04c80d09e0 msgr2=0x7f04c80d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:09.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 -- 172.21.15.89:0/4184590649 shutdown_connections 2023-07-06T02:24:09.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.666+0000 7f04cd71e700 1 -- 172.21.15.89:0/4184590649 wait complete. 2023-07-06T02:24:09.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.667+0000 7f04cd71e700 1 Processor -- start 2023-07-06T02:24:09.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.667+0000 7f04cd71e700 1 -- start start 2023-07-06T02:24:09.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04cd71e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c8070920 0x7f04c80c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:09.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04cd71e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04cd71e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c815c4f0 0x7f04c80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04cd71e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c8155ca0 con 0x7f04c80ccfe0 2023-07-06T02:24:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04cd71e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80cab50 con 0x7f04c815c4f0 2023-07-06T02:24:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04cd71e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80c4640 con 0x7f04c8070920 2023-07-06T02:24:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34684/0 (socket says 172.21.15.89:34684) 2023-07-06T02:24:09.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 -- 172.21.15.89:0/2701294706 learned_addr learned my addr 172.21.15.89:0/2701294706 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 -- 172.21.15.89:0/2701294706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c8070920 msgr2=0x7f04c80c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c8070920 0x7f04c80c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 -- 172.21.15.89:0/2701294706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c815c4f0 msgr2=0x7f04c80ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c77fe700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c815c4f0 0x7f04c80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c815c4f0 0x7f04c80ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 -- 172.21.15.89:0/2701294706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc009550 con 0x7f04c80ccfe0 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.668+0000 7f04c67fc700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80ca0d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f04c80d1980 tx=0x7f04bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:09.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.669+0000 7f04af7fe700 1 -- 172.21.15.89:0/2701294706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc011860 con 0x7f04c80ccfe0 2023-07-06T02:24:09.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.669+0000 7f04af7fe700 1 -- 172.21.15.89:0/2701294706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc01f070 con 0x7f04c80ccfe0 2023-07-06T02:24:09.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.669+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c80c4920 con 0x7f04c80ccfe0 2023-07-06T02:24:09.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.669+0000 7f04af7fe700 1 -- 172.21.15.89:0/2701294706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc011bc0 con 0x7f04c80ccfe0 2023-07-06T02:24:09.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.669+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c80c4ee0 con 0x7f04c80ccfe0 2023-07-06T02:24:09.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.670+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c8074270 con 0x7f04c80ccfe0 2023-07-06T02:24:09.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.670+0000 7f04af7fe700 1 -- 172.21.15.89:0/2701294706 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f04bc021030 con 0x7f04c80ccfe0 2023-07-06T02:24:09.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.671+0000 7f04af7fe700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04b406ace0 0x7f04b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:09.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.671+0000 7f04c6ffd700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04b406ace0 0x7f04b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:09.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.671+0000 7f04af7fe700 1 -- 172.21.15.89:0/2701294706 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f04bc08ab10 con 0x7f04c80ccfe0 2023-07-06T02:24:09.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.671+0000 7f04c6ffd700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04b406ace0 0x7f04b406d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001610 tx=0x7f04b80079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:09.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.675+0000 7f04af7fe700 1 -- 172.21.15.89:0/2701294706 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f04bc055b70 con 0x7f04c80ccfe0 2023-07-06T02:24:09.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:09.905+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f04c80cb0d0 con 0x7f04c80ccfe0 2023-07-06T02:24:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.526+0000 7f04af7fe700 1 -- 172.21.15.89:0/2701294706 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f04bc059190 con 0x7f04c80ccfe0 2023-07-06T02:24:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.531+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04b406ace0 msgr2=0x7f04b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.531+0000 7f04cd71e700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04b406ace0 0x7f04b406d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001610 tx=0x7f04b80079f0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.531+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 msgr2=0x7f04c80ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.531+0000 7f04cd71e700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80ca0d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f04c80d1980 tx=0x7f04bc004750 comp rx=0 tx=0).stop 2023-07-06T02:24:10.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 shutdown_connections 2023-07-06T02:24:10.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c815c4f0 0x7f04c80ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04c8070920 0x7f04c80c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f04b406ace0 0x7f04b406d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 --2- 172.21.15.89:0/2701294706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04c80ccfe0 0x7f04c80ca0d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 >> 172.21.15.89:0/2701294706 conn(0x7f04c80d09e0 msgr2=0x7f04c80b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 shutdown_connections 2023-07-06T02:24:10.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.533+0000 7f04cd71e700 1 -- 172.21.15.89:0/2701294706 wait complete. 2023-07-06T02:24:10.549 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 6 and data pool 7 2023-07-06T02:24:10.567 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:24:10.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.686+0000 7f9465cd1700 1 Processor -- start 2023-07-06T02:24:10.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.686+0000 7f9465cd1700 1 -- start start 2023-07-06T02:24:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f9465cd1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f946015c500 0x7f946006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f9465cd1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9460070f20 0x7f9460071390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f9465cd1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9460160000 0x7f9460160460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f9465cd1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600718d0 con 0x7f9460160000 2023-07-06T02:24:10.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f9465cd1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600cb9f0 con 0x7f9460070f20 2023-07-06T02:24:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f9465cd1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94601609a0 con 0x7f946015c500 2023-07-06T02:24:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f945ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9460160000 0x7f9460160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f945ffff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9460160000 0x7f9460160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34708/0 (socket says 172.21.15.89:34708) 2023-07-06T02:24:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.687+0000 7f945ffff700 1 -- 172.21.15.89:0/1546553525 learned_addr learned my addr 172.21.15.89:0/1546553525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945f7fe700 1 --2- 172.21.15.89:0/1546553525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f946015c500 0x7f946006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945ffff700 1 -- 172.21.15.89:0/1546553525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f946015c500 msgr2=0x7f946006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945effd700 1 --2- 172.21.15.89:0/1546553525 >> v2:172.21.15.172:3300/0 conn(0x7f9460070f20 0x7f9460071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945ffff700 1 --2- 172.21.15.89:0/1546553525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f946015c500 0x7f946006edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945ffff700 1 -- 172.21.15.89:0/1546553525 >> v2:172.21.15.172:3300/0 conn(0x7f9460070f20 msgr2=0x7f9460071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945ffff700 1 --2- 172.21.15.89:0/1546553525 >> v2:172.21.15.172:3300/0 conn(0x7f9460070f20 0x7f9460071390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945ffff700 1 -- 172.21.15.89:0/1546553525 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946006f310 con 0x7f9460160000 2023-07-06T02:24:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945ffff700 1 --2- 172.21.15.89:0/1546553525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f9460160460 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f946006fad0 tx=0x7f9450009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b64b5ad5415a096 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:10.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945e7fc700 1 -- 172.21.15.89:0/1546553525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000b860 con 0x7f9460160000 2023-07-06T02:24:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945e7fc700 1 -- 172.21.15.89:0/1546553525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945000b9c0 con 0x7f9460160000 2023-07-06T02:24:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f945e7fc700 1 -- 172.21.15.89:0/1546553525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000bd50 con 0x7f9460160000 2023-07-06T02:24:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 -- 172.21.15.89:0/1546553525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 msgr2=0x7f9460160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 --2- 172.21.15.89:0/1546553525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f9460160460 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f946006fad0 tx=0x7f9450009c00 comp rx=0 tx=0).stop 2023-07-06T02:24:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 -- 172.21.15.89:0/1546553525 shutdown_connections 2023-07-06T02:24:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 --2- 172.21.15.89:0/1546553525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f946015c500 0x7f946006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 --2- 172.21.15.89:0/1546553525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f9460160460 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 --2- 172.21.15.89:0/1546553525 >> v2:172.21.15.172:3300/0 conn(0x7f9460070f20 0x7f9460071390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 -- 172.21.15.89:0/1546553525 >> 172.21.15.89:0/1546553525 conn(0x7f94600cf1c0 msgr2=0x7f94600cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 -- 172.21.15.89:0/1546553525 shutdown_connections 2023-07-06T02:24:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.688+0000 7f9465cd1700 1 -- 172.21.15.89:0/1546553525 wait complete. 2023-07-06T02:24:10.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 Processor -- start 2023-07-06T02:24:10.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 -- start start 2023-07-06T02:24:10.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9460070f20 0x7f94600afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:10.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946015c500 0x7f94601f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f94600b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600b20a0 con 0x7f9460160000 2023-07-06T02:24:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600b3fd0 con 0x7f946015c500 2023-07-06T02:24:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.690+0000 7f9465cd1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600b4200 con 0x7f9460070f20 2023-07-06T02:24:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f94600b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f94600b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34714/0 (socket says 172.21.15.89:34714) 2023-07-06T02:24:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 -- 172.21.15.89:0/3772093642 learned_addr learned my addr 172.21.15.89:0/3772093642 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945f7fe700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9460070f20 0x7f94600afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945effd700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946015c500 0x7f94601f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 -- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9460070f20 msgr2=0x7f94600afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9460070f20 0x7f94600afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 -- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946015c500 msgr2=0x7f94601f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946015c500 0x7f94601f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 -- 172.21.15.89:0/3772093642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450009550 con 0x7f9460160000 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f945ffff700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f94600b3a90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f94500098f0 tx=0x7f9450004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f9457fff700 1 -- 172.21.15.89:0/3772093642 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450004240 con 0x7f9460160000 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f9457fff700 1 -- 172.21.15.89:0/3772093642 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945001f070 con 0x7f9460160000 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94600b44e0 con 0x7f9460160000 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f9457fff700 1 -- 172.21.15.89:0/3772093642 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500036a0 con 0x7f9460160000 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.691+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94600b4aa0 con 0x7f9460160000 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.692+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94600b2210 con 0x7f9460160000 2023-07-06T02:24:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.693+0000 7f9457fff700 1 -- 172.21.15.89:0/3772093642 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9450021030 con 0x7f9460160000 2023-07-06T02:24:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.693+0000 7f9457fff700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f944006ace0 0x7f944006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.693+0000 7f9457fff700 1 -- 172.21.15.89:0/3772093642 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f945008a920 con 0x7f9460160000 2023-07-06T02:24:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.693+0000 7f945f7fe700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f944006ace0 0x7f944006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.693+0000 7f945f7fe700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f944006ace0 0x7f944006d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94600b1cd0 tx=0x7f94580077b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:10.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.697+0000 7f9457fff700 1 -- 172.21.15.89:0/3772093642 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f94600b2210 con 0x7f9460160000 2023-07-06T02:24:10.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.887+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94600b21e0 con 0x7f9460160000 2023-07-06T02:24:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.889+0000 7f9457fff700 1 -- 172.21.15.89:0/3772093642 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18443 (secure 0 0 0) 0x7f94500558a0 con 0x7f9460160000 2023-07-06T02:24:10.892 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:10.892 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":37,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:24:09.907084+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:24:08.103741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-06T02:24:08.871993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:24:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f944006ace0 msgr2=0x7f944006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f944006ace0 0x7f944006d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94600b1cd0 tx=0x7f94580077b0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 msgr2=0x7f94600b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f94600b3a90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f94500098f0 tx=0x7f9450004750 comp rx=0 tx=0).stop 2023-07-06T02:24:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 shutdown_connections 2023-07-06T02:24:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f944006ace0 0x7f944006d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9460160000 0x7f94600b3a90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9460070f20 0x7f94600afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 --2- 172.21.15.89:0/3772093642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946015c500 0x7f94601f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 >> 172.21.15.89:0/3772093642 conn(0x7f94600cf1c0 msgr2=0x7f94600ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 shutdown_connections 2023-07-06T02:24:10.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:10.891+0000 7f9465cd1700 1 -- 172.21.15.89:0/3772093642 wait complete. 2023-07-06T02:24:10.906 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:24:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.023+0000 7f7ad6cb7700 1 Processor -- start 2023-07-06T02:24:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.023+0000 7f7ad6cb7700 1 -- start start 2023-07-06T02:24:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad6cb7700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ad00afec0 0x7f7ad00702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad6cb7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7ad006e620 0x7f7ad00707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad6cb7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad00cb450 con 0x7f7ad00d0560 2023-07-06T02:24:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad6cb7700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad00cb1b0 con 0x7f7ad00afec0 2023-07-06T02:24:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad6cb7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad006ebc0 con 0x7f7ad006e620 2023-07-06T02:24:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad4a53700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ad00afec0 0x7f7ad00702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad4a53700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ad00afec0 0x7f7ad00702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34718/0 (socket says 172.21.15.89:34718) 2023-07-06T02:24:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad4a53700 1 -- 172.21.15.89:0/4201760538 learned_addr learned my addr 172.21.15.89:0/4201760538 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.024+0000 7f7ad5254700 1 --2- 172.21.15.89:0/4201760538 >> v2:172.21.15.172:3300/0 conn(0x7f7ad006e620 0x7f7ad00707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad4a53700 1 -- 172.21.15.89:0/4201760538 >> v2:172.21.15.172:3300/0 conn(0x7f7ad006e620 msgr2=0x7f7ad00707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad4a53700 1 --2- 172.21.15.89:0/4201760538 >> v2:172.21.15.172:3300/0 conn(0x7f7ad006e620 0x7f7ad00707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad4a53700 1 -- 172.21.15.89:0/4201760538 >> v1:172.21.15.172:6789/0 conn(0x7f7ad00d0560 legacy=0x7f7ad015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad4a53700 1 -- 172.21.15.89:0/4201760538 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad00ccc10 con 0x7f7ad00afec0 2023-07-06T02:24:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad4a53700 1 --2- 172.21.15.89:0/4201760538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad00702a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0071200 tx=0x7f7ac4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe5947d535be665 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7acf7fe700 1 -- 172.21.15.89:0/4201760538 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4011830 con 0x7f7ad00afec0 2023-07-06T02:24:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7acf7fe700 1 -- 172.21.15.89:0/4201760538 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac4011990 con 0x7f7ad00afec0 2023-07-06T02:24:11.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7acf7fe700 1 -- 172.21.15.89:0/4201760538 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4011d20 con 0x7f7ad00afec0 2023-07-06T02:24:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/4201760538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 msgr2=0x7f7ad00702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/4201760538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad00702a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0071200 tx=0x7f7ac4009bd0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/4201760538 shutdown_connections 2023-07-06T02:24:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.025+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/4201760538 >> v2:172.21.15.172:3300/0 conn(0x7f7ad006e620 0x7f7ad00707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.026+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/4201760538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad00702a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.026+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/4201760538 >> 172.21.15.89:0/4201760538 conn(0x7f7ad00d0ad0 msgr2=0x7f7ad00d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.026+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/4201760538 shutdown_connections 2023-07-06T02:24:11.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.026+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/4201760538 wait complete. 2023-07-06T02:24:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.027+0000 7f7ad6cb7700 1 Processor -- start 2023-07-06T02:24:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.027+0000 7f7ad6cb7700 1 -- start start 2023-07-06T02:24:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad006e620 0x7f7ad0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad00d0560 0x7f7ad00c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00ada30 con 0x7f7ad00afec0 2023-07-06T02:24:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00ca790 con 0x7f7ad006e620 2023-07-06T02:24:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00c8b40 con 0x7f7ad00d0560 2023-07-06T02:24:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad006ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34726/0 (socket says 172.21.15.89:34726) 2023-07-06T02:24:11.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 -- 172.21.15.89:0/2221791499 learned_addr learned my addr 172.21.15.89:0/2221791499 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad4a53700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad006e620 0x7f7ad0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad5254700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad00d0560 0x7f7ad00c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 -- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad00d0560 msgr2=0x7f7ad00c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad00d0560 0x7f7ad00c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 -- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad006e620 msgr2=0x7f7ad0151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad006e620 0x7f7ad0151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 -- 172.21.15.89:0/2221791499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac4009550 con 0x7f7ad00afec0 2023-07-06T02:24:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acffff700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad006ba00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c4e0 tx=0x7f7abc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acd7fa700 1 -- 172.21.15.89:0/2221791499 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0052d0 con 0x7f7ad00afec0 2023-07-06T02:24:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acd7fa700 1 -- 172.21.15.89:0/2221791499 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc005430 con 0x7f7ad00afec0 2023-07-06T02:24:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad00adc60 con 0x7f7ad00afec0 2023-07-06T02:24:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7acd7fa700 1 -- 172.21.15.89:0/2221791499 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0038c0 con 0x7f7ad00afec0 2023-07-06T02:24:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.028+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad00ae220 con 0x7f7ad00afec0 2023-07-06T02:24:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.029+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad0074270 con 0x7f7ad00afec0 2023-07-06T02:24:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.030+0000 7f7acd7fa700 1 -- 172.21.15.89:0/2221791499 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7abc003a20 con 0x7f7ad00afec0 2023-07-06T02:24:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.030+0000 7f7acd7fa700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ab406ace0 0x7f7ab406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.030+0000 7f7acd7fa700 1 -- 172.21.15.89:0/2221791499 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7abc088930 con 0x7f7ad00afec0 2023-07-06T02:24:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.030+0000 7f7ad4a53700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ab406ace0 0x7f7ab406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.031+0000 7f7ad4a53700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ab406ace0 0x7f7ab406d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4000c00 tx=0x7f7ac4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.035+0000 7f7acd7fa700 1 -- 172.21.15.89:0/2221791499 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7abc053890 con 0x7f7ad00afec0 2023-07-06T02:24:11.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.226+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ad006f4e0 con 0x7f7ad00afec0 2023-07-06T02:24:11.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.227+0000 7f7acd7fa700 1 -- 172.21.15.89:0/2221791499 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18443 (secure 0 0 0) 0x7f7abc056eb0 con 0x7f7ad00afec0 2023-07-06T02:24:11.230 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:11.230 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":37,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:24:09.907084+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:24:08.103741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-06T02:24:08.871993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:24:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ab406ace0 msgr2=0x7f7ab406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ab406ace0 0x7f7ab406d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4000c00 tx=0x7f7ac4008030 comp rx=0 tx=0).stop 2023-07-06T02:24:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 msgr2=0x7f7ad006ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad006ba00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c4e0 tx=0x7f7abc00c7f0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 shutdown_connections 2023-07-06T02:24:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad00d0560 0x7f7ad00c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad006e620 0x7f7ad0151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f7ab406ace0 0x7f7ab406d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 --2- 172.21.15.89:0/2221791499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad00afec0 0x7f7ad006ba00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 >> 172.21.15.89:0/2221791499 conn(0x7f7ad00d0ad0 msgr2=0x7f7ad00b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 shutdown_connections 2023-07-06T02:24:11.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.229+0000 7f7ad6cb7700 1 -- 172.21.15.89:0/2221791499 wait complete. 2023-07-06T02:24:11.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T02:24:11.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.363+0000 7ff188247700 1 Processor -- start 2023-07-06T02:24:11.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.364+0000 7ff188247700 1 -- start start 2023-07-06T02:24:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.364+0000 7ff188247700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff180156ab0 0x7ff180160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.364+0000 7ff188247700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1801600b0 0x7ff1800cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.364+0000 7ff188247700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1800cb3d0 con 0x7ff180160bb0 2023-07-06T02:24:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.364+0000 7ff188247700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff180156e80 con 0x7ff1801600b0 2023-07-06T02:24:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.364+0000 7ff188247700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800ca370 con 0x7ff180156ab0 2023-07-06T02:24:11.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1801600b0 0x7ff1800cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff1801600b0 0x7ff1800cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34734/0 (socket says 172.21.15.89:34734) 2023-07-06T02:24:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 -- 172.21.15.89:0/4002590936 learned_addr learned my addr 172.21.15.89:0/4002590936 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff185fe3700 1 --2- 172.21.15.89:0/4002590936 >> v2:172.21.15.172:3300/0 conn(0x7ff180156ab0 0x7ff180160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 -- 172.21.15.89:0/4002590936 >> v2:172.21.15.172:3300/0 conn(0x7ff180156ab0 msgr2=0x7ff180160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 --2- 172.21.15.89:0/4002590936 >> v2:172.21.15.172:3300/0 conn(0x7ff180156ab0 0x7ff180160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 -- 172.21.15.89:0/4002590936 >> v1:172.21.15.172:6789/0 conn(0x7ff180160bb0 legacy=0x7ff1800b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:11.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 -- 172.21.15.89:0/4002590936 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800ca610 con 0x7ff1801600b0 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff185fe3700 1 --2- 172.21.15.89:0/4002590936 >> v2:172.21.15.172:3300/0 conn(0x7ff180156ab0 0x7ff180160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff1867e4700 1 --2- 172.21.15.89:0/4002590936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1801600b0 0x7ff1800cd700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff18006e9e0 tx=0x7ff17800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb9e50ec3f4586f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff184fe1700 1 -- 172.21.15.89:0/4002590936 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17800e070 con 0x7ff1801600b0 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff184fe1700 1 -- 172.21.15.89:0/4002590936 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff178003a20 con 0x7ff1801600b0 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.365+0000 7ff184fe1700 1 -- 172.21.15.89:0/4002590936 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178003dd0 con 0x7ff1801600b0 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 -- 172.21.15.89:0/4002590936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1801600b0 msgr2=0x7ff1800cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 --2- 172.21.15.89:0/4002590936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1801600b0 0x7ff1800cd700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff18006e9e0 tx=0x7ff17800b330 comp rx=0 tx=0).stop 2023-07-06T02:24:11.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 -- 172.21.15.89:0/4002590936 shutdown_connections 2023-07-06T02:24:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 --2- 172.21.15.89:0/4002590936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1801600b0 0x7ff1800cd700 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 --2- 172.21.15.89:0/4002590936 >> v2:172.21.15.172:3300/0 conn(0x7ff180156ab0 0x7ff180160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 -- 172.21.15.89:0/4002590936 >> 172.21.15.89:0/4002590936 conn(0x7ff1800cc490 msgr2=0x7ff1800cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 -- 172.21.15.89:0/4002590936 shutdown_connections 2023-07-06T02:24:11.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.366+0000 7ff188247700 1 -- 172.21.15.89:0/4002590936 wait complete. 2023-07-06T02:24:11.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.367+0000 7ff188247700 1 Processor -- start 2023-07-06T02:24:11.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.367+0000 7ff188247700 1 -- start start 2023-07-06T02:24:11.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff188247700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff180156ab0 0x7ff18014feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff188247700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1801600b0 0x7ff1800ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff188247700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1800ab8f0 0x7ff1800ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff188247700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800afef0 con 0x7ff1800ab8f0 2023-07-06T02:24:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff188247700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800b0190 con 0x7ff1801600b0 2023-07-06T02:24:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff188247700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800b0470 con 0x7ff180156ab0 2023-07-06T02:24:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1800ab8f0 0x7ff1800ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1800ab8f0 0x7ff1800ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34740/0 (socket says 172.21.15.89:34740) 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 -- 172.21.15.89:0/1120905643 learned_addr learned my addr 172.21.15.89:0/1120905643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 -- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff180156ab0 msgr2=0x7ff18014feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1857e2700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1801600b0 0x7ff1800ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff185fe3700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff180156ab0 0x7ff18014feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff180156ab0 0x7ff18014feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 -- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1801600b0 msgr2=0x7ff1800ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1801600b0 0x7ff1800ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:11.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 -- 172.21.15.89:0/1120905643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff178009b30 con 0x7ff1800ab8f0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.368+0000 7ff1867e4700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1800ab8f0 0x7ff1800ad8e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff178015040 tx=0x7ff178004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.369+0000 7ff1767fc700 1 -- 172.21.15.89:0/1120905643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178004440 con 0x7ff1800ab8f0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.369+0000 7ff1767fc700 1 -- 172.21.15.89:0/1120905643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1780045a0 con 0x7ff1800ab8f0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.369+0000 7ff1767fc700 1 -- 172.21.15.89:0/1120905643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178004950 con 0x7ff1800ab8f0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.369+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1800b0750 con 0x7ff1800ab8f0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.369+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1801503f0 con 0x7ff1800ab8f0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.370+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1800b0990 con 0x7ff1800ab8f0 2023-07-06T02:24:11.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.370+0000 7ff1767fc700 1 -- 172.21.15.89:0/1120905643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff178012430 con 0x7ff1800ab8f0 2023-07-06T02:24:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.370+0000 7ff1767fc700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff16406ace0 0x7ff16406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.371+0000 7ff1767fc700 1 -- 172.21.15.89:0/1120905643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff17808ac50 con 0x7ff1800ab8f0 2023-07-06T02:24:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.371+0000 7ff185fe3700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff16406ace0 0x7ff16406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.371+0000 7ff185fe3700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff16406ace0 0x7ff16406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff170006fd0 tx=0x7ff170008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:11.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.375+0000 7ff1767fc700 1 -- 172.21.15.89:0/1120905643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff1800b0990 con 0x7ff1800ab8f0 2023-07-06T02:24:11.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:11.611+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff180074270 con 0x7ff1800ab8f0 2023-07-06T02:24:12.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.543+0000 7ff1767fc700 1 -- 172.21.15.89:0/1120905643 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff1780595c0 con 0x7ff1800ab8f0 2023-07-06T02:24:12.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff16406ace0 msgr2=0x7ff16406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff16406ace0 0x7ff16406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff170006fd0 tx=0x7ff170008040 comp rx=0 tx=0).stop 2023-07-06T02:24:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1800ab8f0 msgr2=0x7ff1800ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1800ab8f0 0x7ff1800ad8e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff178015040 tx=0x7ff178004ad0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 shutdown_connections 2023-07-06T02:24:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff16406ace0 0x7ff16406d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1800ab8f0 0x7ff1800ad8e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff180156ab0 0x7ff18014feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 --2- 172.21.15.89:0/1120905643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1801600b0 0x7ff1800ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.546+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 >> 172.21.15.89:0/1120905643 conn(0x7ff1800cc490 msgr2=0x7ff18015ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.547+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 shutdown_connections 2023-07-06T02:24:12.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.547+0000 7ff188247700 1 -- 172.21.15.89:0/1120905643 wait complete. 2023-07-06T02:24:12.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T02:24:12.564 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T02:24:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.691+0000 7f2419ab3700 1 Processor -- start 2023-07-06T02:24:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.691+0000 7f2419ab3700 1 -- start start 2023-07-06T02:24:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2419ab3700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f24140b1490 0x7f24140b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2419ab3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f241406bfa0 0x7f241406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2419ab3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140cab50 con 0x7f241415c690 2023-07-06T02:24:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2419ab3700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241415de50 con 0x7f24140b1490 2023-07-06T02:24:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2419ab3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24140b1e50 con 0x7f241406bfa0 2023-07-06T02:24:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f24140b1490 0x7f24140b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f24140b1490 0x7f24140b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34746/0 (socket says 172.21.15.89:34746) 2023-07-06T02:24:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 -- 172.21.15.89:0/1286886854 learned_addr learned my addr 172.21.15.89:0/1286886854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2413fff700 1 --2- 172.21.15.89:0/1286886854 >> v2:172.21.15.172:3300/0 conn(0x7f241406bfa0 0x7f241406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 -- 172.21.15.89:0/1286886854 >> v2:172.21.15.172:3300/0 conn(0x7f241406bfa0 msgr2=0x7f241406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 --2- 172.21.15.89:0/1286886854 >> v2:172.21.15.172:3300/0 conn(0x7f241406bfa0 0x7f241406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 -- 172.21.15.89:0/1286886854 >> v1:172.21.15.172:6789/0 conn(0x7f241415c690 legacy=0x7f24140cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 -- 172.21.15.89:0/1286886854 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241406c8e0 con 0x7f24140b1490 2023-07-06T02:24:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2413fff700 1 --2- 172.21.15.89:0/1286886854 >> v2:172.21.15.172:3300/0 conn(0x7f241406bfa0 0x7f241406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f2412ffd700 1 --2- 172.21.15.89:0/1286886854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140b1910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f24140af420 tx=0x7f2408009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103975fd97fd2612 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f24127fc700 1 -- 172.21.15.89:0/1286886854 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408004230 con 0x7f24140b1490 2023-07-06T02:24:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f24127fc700 1 -- 172.21.15.89:0/1286886854 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2408004390 con 0x7f24140b1490 2023-07-06T02:24:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.692+0000 7f24127fc700 1 -- 172.21.15.89:0/1286886854 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24080036a0 con 0x7f24140b1490 2023-07-06T02:24:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 -- 172.21.15.89:0/1286886854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 msgr2=0x7f24140b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 --2- 172.21.15.89:0/1286886854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140b1910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f24140af420 tx=0x7f2408009bd0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 -- 172.21.15.89:0/1286886854 shutdown_connections 2023-07-06T02:24:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 --2- 172.21.15.89:0/1286886854 >> v2:172.21.15.172:3300/0 conn(0x7f241406bfa0 0x7f241406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 --2- 172.21.15.89:0/1286886854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140b1910 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 -- 172.21.15.89:0/1286886854 >> 172.21.15.89:0/1286886854 conn(0x7f24140cc490 msgr2=0x7f24140cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 -- 172.21.15.89:0/1286886854 shutdown_connections 2023-07-06T02:24:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.693+0000 7f2419ab3700 1 -- 172.21.15.89:0/1286886854 wait complete. 2023-07-06T02:24:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.694+0000 7f2419ab3700 1 Processor -- start 2023-07-06T02:24:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.694+0000 7f2419ab3700 1 -- start start 2023-07-06T02:24:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2419ab3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f241406bfa0 0x7f241415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2419ab3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2419ab3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f241415c690 0x7f241415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2419ab3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24140c63a0 con 0x7f24140b1490 2023-07-06T02:24:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2419ab3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24140ac970 con 0x7f241415c690 2023-07-06T02:24:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2419ab3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24140acc50 con 0x7f241406bfa0 2023-07-06T02:24:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34752/0 (socket says 172.21.15.89:34752) 2023-07-06T02:24:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 -- 172.21.15.89:0/2554683834 learned_addr learned my addr 172.21.15.89:0/2554683834 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f24137fe700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f241406bfa0 0x7f241415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2413fff700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f241415c690 0x7f241415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 -- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f241406bfa0 msgr2=0x7f241415a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f241406bfa0 0x7f241415a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 -- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f241415c690 msgr2=0x7f241415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f241415c690 0x7f241415ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 -- 172.21.15.89:0/2554683834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2408009550 con 0x7f24140b1490 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f24137fe700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f241406bfa0 0x7f241415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.695+0000 7f2412ffd700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140ac2a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f241406f2b0 tx=0x7f2408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.696+0000 7f23fbfff700 1 -- 172.21.15.89:0/2554683834 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408003d80 con 0x7f24140b1490 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.696+0000 7f23fbfff700 1 -- 172.21.15.89:0/2554683834 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240801f070 con 0x7f24140b1490 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.696+0000 7f23fbfff700 1 -- 172.21.15.89:0/2554683834 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408011870 con 0x7f24140b1490 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.696+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24140acf30 con 0x7f24140b1490 2023-07-06T02:24:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.696+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24140ad4f0 con 0x7f24140b1490 2023-07-06T02:24:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.697+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24140b2e00 con 0x7f24140b1490 2023-07-06T02:24:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.697+0000 7f23fbfff700 1 -- 172.21.15.89:0/2554683834 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2408021030 con 0x7f24140b1490 2023-07-06T02:24:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.697+0000 7f23fbfff700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f240006ace0 0x7f240006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.698+0000 7f23fbfff700 1 -- 172.21.15.89:0/2554683834 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f240808ac80 con 0x7f24140b1490 2023-07-06T02:24:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.698+0000 7f24137fe700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f240006ace0 0x7f240006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.698+0000 7f24137fe700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f240006ace0 0x7f240006d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2404003fc0 tx=0x7f2404009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:12.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.702+0000 7f23fbfff700 1 -- 172.21.15.89:0/2554683834 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2408055c80 con 0x7f24140b1490 2023-07-06T02:24:12.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:12.934+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f24140ad6d0 con 0x7f24140b1490 2023-07-06T02:24:13.475 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-07-06T02:24:13.472+0000 7f27acae8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T02:24:13.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.551+0000 7f23fbfff700 1 -- 172.21.15.89:0/2554683834 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f24080592a0 con 0x7f24140b1490 2023-07-06T02:24:13.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f240006ace0 msgr2=0x7f240006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f240006ace0 0x7f240006d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2404003fc0 tx=0x7f2404009040 comp rx=0 tx=0).stop 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 msgr2=0x7f24140ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140ac2a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f241406f2b0 tx=0x7f2408004750 comp rx=0 tx=0).stop 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 shutdown_connections 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f241415c690 0x7f241415ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f241406bfa0 0x7f241415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f240006ace0 0x7f240006d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 --2- 172.21.15.89:0/2554683834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24140b1490 0x7f24140ac2a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.553+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 >> 172.21.15.89:0/2554683834 conn(0x7f24140cc490 msgr2=0x7f241415a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:13.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.554+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 shutdown_connections 2023-07-06T02:24:13.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.554+0000 7f2419ab3700 1 -- 172.21.15.89:0/2554683834 wait complete. 2023-07-06T02:24:13.571 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T02:24:13.571 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:24:13.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.694+0000 7fd52bf3d700 1 Processor -- start 2023-07-06T02:24:13.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.694+0000 7fd52bf3d700 1 -- start start 2023-07-06T02:24:13.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd52bf3d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5240b5e50 0x7fd524070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd52bf3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 0x7fd5240cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd52bf3d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240cb150 con 0x7fd52415c610 2023-07-06T02:24:13.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd52bf3d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5240d05e0 con 0x7fd5240b5e50 2023-07-06T02:24:13.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd52bf3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240cb430 con 0x7fd524070870 2023-07-06T02:24:13.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd529cd9700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd52415c610 0x7fd5240cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59660/0 (socket says 172.21.15.89:59660) 2023-07-06T02:24:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd529cd9700 1 -- 172.21.15.89:0/288471207 learned_addr learned my addr 172.21.15.89:0/288471207 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613046033 0 0) 0x7fd5240cb150 con 0x7fd52415c610 2023-07-06T02:24:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5100035b0 con 0x7fd52415c610 2023-07-06T02:24:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd52a4da700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 0x7fd5240cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd5294d8700 1 --2- 172.21.15.89:0/288471207 >> v2:172.21.15.172:3300/0 conn(0x7fd5240b5e50 0x7fd524070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483124501 0 0) 0x7fd5100035b0 con 0x7fd52415c610 2023-07-06T02:24:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 msgr2=0x7fd5240cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 0x7fd5240cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 >> v2:172.21.15.172:3300/0 conn(0x7fd5240b5e50 msgr2=0x7fd524070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 --2- 172.21.15.89:0/288471207 >> v2:172.21.15.172:3300/0 conn(0x7fd5240b5e50 0x7fd524070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd524070cb0 con 0x7fd52415c610 2023-07-06T02:24:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7fd520001670 con 0x7fd52415c610 2023-07-06T02:24:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 >> v1:172.21.15.89:6789/0 conn(0x7fd52415c610 legacy=0x7fd5240cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 0x7fd510003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd510003f80 0x7fd510004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd510004d90 0x7fd510005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240d05e0 con 0x7fd510003f80 2023-07-06T02:24:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240cb430 con 0x7fd510004d90 2023-07-06T02:24:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51000ba10 con 0x7fd524070870 2023-07-06T02:24:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.695+0000 7fd5294d8700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd510003f80 0x7fd510004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd529cd9700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 0x7fd510003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd52a4da700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd510004d90 0x7fd510005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd5294d8700 1 -- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 msgr2=0x7fd510003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd5294d8700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 0x7fd510003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd5294d8700 1 -- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd510004d90 msgr2=0x7fd510005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd5294d8700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd510004d90 0x7fd510005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd5294d8700 1 -- 172.21.15.89:0/288471207 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100035b0 con 0x7fd510003f80 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd529cd9700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd524070870 0x7fd510003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd5294d8700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd510003f80 0x7fd510004850 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd514009b60 tx=0x7fd510005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74caca18880ab4a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400e6c0 con 0x7fd510003f80 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400e820 con 0x7fd510003f80 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd528cd7700 1 -- 172.21.15.89:0/288471207 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400eb90 con 0x7fd510003f80 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.696+0000 7fd52bf3d700 1 -- 172.21.15.89:0/288471207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd510003f80 msgr2=0x7fd510004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.697+0000 7fd52bf3d700 1 --2- 172.21.15.89:0/288471207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd510003f80 0x7fd510004850 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd514009b60 tx=0x7fd510005880 comp rx=0 tx=0).stop 2023-07-06T02:24:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.697+0000 7fd52a4da700 1 -- 172.21.15.89:0/288471207 reap_dead start 2023-07-06T02:24:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.697+0000 7fd52bf3d700 1 -- 172.21.15.89:0/288471207 shutdown_connections 2023-07-06T02:24:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.697+0000 7fd52bf3d700 1 -- 172.21.15.89:0/288471207 >> 172.21.15.89:0/288471207 conn(0x7fd5240d0b00 msgr2=0x7fd5240d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.697+0000 7fd52bf3d700 1 -- 172.21.15.89:0/288471207 shutdown_connections 2023-07-06T02:24:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.697+0000 7fd52bf3d700 1 -- 172.21.15.89:0/288471207 wait complete. 2023-07-06T02:24:13.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.698+0000 7fd52bf3d700 1 Processor -- start 2023-07-06T02:24:13.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.698+0000 7fd52bf3d700 1 -- start start 2023-07-06T02:24:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52bf3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd510003f80 0x7fd5240ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52bf3d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5240c6bc0 0x7fd5240c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52bf3d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd524148b80 0x7fd524148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52bf3d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd524149530 con 0x7fd524148b80 2023-07-06T02:24:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52bf3d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5241f53c0 con 0x7fd5240c6bc0 2023-07-06T02:24:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52bf3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5241f56a0 con 0x7fd510003f80 2023-07-06T02:24:13.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52a4da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd524148b80 0x7fd524148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52a4da700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd524148b80 0x7fd524148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34774/0 (socket says 172.21.15.89:34774) 2023-07-06T02:24:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.699+0000 7fd52a4da700 1 -- 172.21.15.89:0/1059948056 learned_addr learned my addr 172.21.15.89:0/1059948056 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd529cd9700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd510003f80 0x7fd5240ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd5294d8700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5240c6bc0 0x7fd5240c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52a4da700 1 -- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd510003f80 msgr2=0x7fd5240ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52a4da700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd510003f80 0x7fd5240ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52a4da700 1 -- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5240c6bc0 msgr2=0x7fd5240c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52a4da700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5240c6bc0 0x7fd5240c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52a4da700 1 -- 172.21.15.89:0/1059948056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5241f5980 con 0x7fd524148b80 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd529cd9700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd510003f80 0x7fd5240ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd5294d8700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5240c6bc0 0x7fd5240c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52a4da700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd524148b80 0x7fd524148ff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5240d0ad0 tx=0x7fd51c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd51a7fc700 1 -- 172.21.15.89:0/1059948056 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00e040 con 0x7fd524148b80 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd51a7fc700 1 -- 172.21.15.89:0/1059948056 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c00f040 con 0x7fd524148b80 2023-07-06T02:24:13.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5241f5c60 con 0x7fd524148b80 2023-07-06T02:24:13.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd51a7fc700 1 -- 172.21.15.89:0/1059948056 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00fdb0 con 0x7fd524148b80 2023-07-06T02:24:13.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.700+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5241f6220 con 0x7fd524148b80 2023-07-06T02:24:13.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.701+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd524074270 con 0x7fd524148b80 2023-07-06T02:24:13.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.701+0000 7fd51a7fc700 1 -- 172.21.15.89:0/1059948056 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd51c022460 con 0x7fd524148b80 2023-07-06T02:24:13.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.701+0000 7fd51a7fc700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd50806ace0 0x7fd50806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:13.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.702+0000 7fd51a7fc700 1 -- 172.21.15.89:0/1059948056 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd51c089ab0 con 0x7fd524148b80 2023-07-06T02:24:13.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.702+0000 7fd529cd9700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd50806ace0 0x7fd50806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:13.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.702+0000 7fd529cd9700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd50806ace0 0x7fd50806d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd520001eb0 tx=0x7fd520001e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:13.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.706+0000 7fd51a7fc700 1 -- 172.21.15.89:0/1059948056 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd51c054a20 con 0x7fd524148b80 2023-07-06T02:24:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.934+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5240d1110 con 0x7fd524148b80 2023-07-06T02:24:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.935+0000 7fd51a7fc700 1 -- 172.21.15.89:0/1059948056 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4668 (secure 0 0 0) 0x7fd51c058040 con 0x7fd524148b80 2023-07-06T02:24:13.938 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:13.938 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/34790451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":34790451},{"type":"v1","addr":"172.21.15.172:6835","nonce":34790451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3141331417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3141331417},{"type":"v1","addr":"172.21.15.172:6834","nonce":3141331417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2965598582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2965598582},{"type":"v1","addr":"172.21.15.89:6836","nonce":2965598582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:24:09.907406+0000","modified":"2023-07-06T02:24:13.543375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5469":{"gid":5469,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/326075892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":326075892},{"type":"v1","addr":"172.21.15.89:6837","nonce":326075892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T02:24:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd50806ace0 msgr2=0x7fd50806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd50806ace0 0x7fd50806d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd520001eb0 tx=0x7fd520001e40 comp rx=0 tx=0).stop 2023-07-06T02:24:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd524148b80 msgr2=0x7fd524148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd524148b80 0x7fd524148ff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5240d0ad0 tx=0x7fd51c011f80 comp rx=0 tx=0).stop 2023-07-06T02:24:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 shutdown_connections 2023-07-06T02:24:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fd50806ace0 0x7fd50806d190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd524148b80 0x7fd524148ff0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd510003f80 0x7fd5240ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 --2- 172.21.15.89:0/1059948056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5240c6bc0 0x7fd5240c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 >> 172.21.15.89:0/1059948056 conn(0x7fd5240d0b00 msgr2=0x7fd5240b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 shutdown_connections 2023-07-06T02:24:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:13.937+0000 7fd52bf3d700 1 -- 172.21.15.89:0/1059948056 wait complete. 2023-07-06T02:24:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2023-07-06T02:24:13.952 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:24:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.069+0000 7fe8c745e700 1 Processor -- start 2023-07-06T02:24:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.070+0000 7fe8c745e700 1 -- start start 2023-07-06T02:24:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.070+0000 7fe8c745e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c0159fd0 0x7fe8c00cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.070+0000 7fe8c745e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe8c0160b80 0x7fe8c006bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c745e700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c00d1a60 con 0x7fe8c006c410 2023-07-06T02:24:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c745e700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c006c890 con 0x7fe8c0160b80 2023-07-06T02:24:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c745e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0160ff0 con 0x7fe8c0159fd0 2023-07-06T02:24:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c49f9700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe8c0160b80 0x7fe8c006bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c59fb700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe8c006c410 0x7fe8c006a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59680/0 (socket says 172.21.15.89:59680) 2023-07-06T02:24:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c59fb700 1 -- 172.21.15.89:0/2570787931 learned_addr learned my addr 172.21.15.89:0/2570787931 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c51fa700 1 --2- 172.21.15.89:0/2570787931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c0159fd0 0x7fe8c00cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c49f9700 1 -- 172.21.15.89:0/2570787931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c0159fd0 msgr2=0x7fe8c00cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c49f9700 1 --2- 172.21.15.89:0/2570787931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c0159fd0 0x7fe8c00cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c49f9700 1 -- 172.21.15.89:0/2570787931 >> v1:172.21.15.89:6789/0 conn(0x7fe8c006c410 legacy=0x7fe8c006a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8b7fff700 1 -- 172.21.15.89:0/2570787931 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222734525 0 0) 0x7fe8c00d1a60 con 0x7fe8c006c410 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c49f9700 1 -- 172.21.15.89:0/2570787931 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00cdb60 con 0x7fe8c0160b80 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c51fa700 1 --2- 172.21.15.89:0/2570787931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c0159fd0 0x7fe8c00cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8c49f9700 1 --2- 172.21.15.89:0/2570787931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0160b80 0x7fe8c006bed0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009840 tx=0x7fe8bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9826cad25502bff6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8b7fff700 1 -- 172.21.15.89:0/2570787931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc011830 con 0x7fe8c0160b80 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8b7fff700 1 -- 172.21.15.89:0/2570787931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc011990 con 0x7fe8c0160b80 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.071+0000 7fe8b7fff700 1 -- 172.21.15.89:0/2570787931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc011d20 con 0x7fe8c0160b80 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 -- 172.21.15.89:0/2570787931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0160b80 msgr2=0x7fe8c006bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 --2- 172.21.15.89:0/2570787931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0160b80 0x7fe8c006bed0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009840 tx=0x7fe8bc009b50 comp rx=0 tx=0).stop 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 -- 172.21.15.89:0/2570787931 shutdown_connections 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 --2- 172.21.15.89:0/2570787931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c0159fd0 0x7fe8c00cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 --2- 172.21.15.89:0/2570787931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0160b80 0x7fe8c006bed0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 -- 172.21.15.89:0/2570787931 >> 172.21.15.89:0/2570787931 conn(0x7fe8c00d0590 msgr2=0x7fe8c00d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 -- 172.21.15.89:0/2570787931 shutdown_connections 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.072+0000 7fe8c745e700 1 -- 172.21.15.89:0/2570787931 wait complete. 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.073+0000 7fe8c745e700 1 Processor -- start 2023-07-06T02:24:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.073+0000 7fe8c745e700 1 -- start start 2023-07-06T02:24:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c745e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c006c410 0x7fe8c014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c745e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0159fd0 0x7fe8c00c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c745e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c0160b80 0x7fe8c00c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c745e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00c6f30 con 0x7fe8c0159fd0 2023-07-06T02:24:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c745e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00c9bf0 con 0x7fe8c0160b80 2023-07-06T02:24:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c745e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00b8bc0 con 0x7fe8c006c410 2023-07-06T02:24:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0159fd0 0x7fe8c00c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0159fd0 0x7fe8c00c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34786/0 (socket says 172.21.15.89:34786) 2023-07-06T02:24:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 -- 172.21.15.89:0/1259927163 learned_addr learned my addr 172.21.15.89:0/1259927163 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c59fb700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c0160b80 0x7fe8c00c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c51fa700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c006c410 0x7fe8c014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 -- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c006c410 msgr2=0x7fe8c014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c006c410 0x7fe8c014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 -- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c0160b80 msgr2=0x7fe8c00c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c0160b80 0x7fe8c00c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 -- 172.21.15.89:0/1259927163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0094a0 con 0x7fe8c0159fd0 2023-07-06T02:24:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.074+0000 7fe8c49f9700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0159fd0 0x7fe8c00c64b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc014040 tx=0x7fe8bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.075+0000 7fe8b5ffb700 1 -- 172.21.15.89:0/1259927163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0038a0 con 0x7fe8c0159fd0 2023-07-06T02:24:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.075+0000 7fe8b5ffb700 1 -- 172.21.15.89:0/1259927163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc01f070 con 0x7fe8c0159fd0 2023-07-06T02:24:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.075+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c00c8fe0 con 0x7fe8c0159fd0 2023-07-06T02:24:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.075+0000 7fe8b5ffb700 1 -- 172.21.15.89:0/1259927163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc003c60 con 0x7fe8c0159fd0 2023-07-06T02:24:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.075+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c00c95a0 con 0x7fe8c0159fd0 2023-07-06T02:24:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.076+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c0074270 con 0x7fe8c0159fd0 2023-07-06T02:24:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.076+0000 7fe8b5ffb700 1 -- 172.21.15.89:0/1259927163 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8bc01a430 con 0x7fe8c0159fd0 2023-07-06T02:24:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.076+0000 7fe8b5ffb700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe8a406ace0 0x7fe8a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.077+0000 7fe8b5ffb700 1 -- 172.21.15.89:0/1259927163 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe8bc021070 con 0x7fe8c0159fd0 2023-07-06T02:24:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.077+0000 7fe8c51fa700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe8a406ace0 0x7fe8a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.077+0000 7fe8c51fa700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe8a406ace0 0x7fe8a406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0006fd0 tx=0x7fe8b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.081+0000 7fe8b5ffb700 1 -- 172.21.15.89:0/1259927163 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe8bc0560d0 con 0x7fe8c0159fd0 2023-07-06T02:24:14.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.270+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8c006b480 con 0x7fe8c0159fd0 2023-07-06T02:24:14.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.271+0000 7fe8b5ffb700 1 -- 172.21.15.89:0/1259927163 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fe8bc0596f0 con 0x7fe8c0159fd0 2023-07-06T02:24:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:14.275 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":38,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:24:11.532935+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:24:08.103741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-06T02:24:08.871993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:24:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe8a406ace0 msgr2=0x7fe8a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe8a406ace0 0x7fe8a406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0006fd0 tx=0x7fe8b0008040 comp rx=0 tx=0).stop 2023-07-06T02:24:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0159fd0 msgr2=0x7fe8c00c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0159fd0 0x7fe8c00c64b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc014040 tx=0x7fe8bc004750 comp rx=0 tx=0).stop 2023-07-06T02:24:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 shutdown_connections 2023-07-06T02:24:14.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c0160b80 0x7fe8c00c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8c006c410 0x7fe8c014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fe8a406ace0 0x7fe8a406d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 --2- 172.21.15.89:0/1259927163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8c0159fd0 0x7fe8c00c64b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 >> 172.21.15.89:0/1259927163 conn(0x7fe8c00d0590 msgr2=0x7fe8c00d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 shutdown_connections 2023-07-06T02:24:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.274+0000 7fe8c745e700 1 -- 172.21.15.89:0/1259927163 wait complete. 2023-07-06T02:24:14.289 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T02:24:14.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T02:24:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.406+0000 7ff4539d1700 1 Processor -- start 2023-07-06T02:24:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.406+0000 7ff4539d1700 1 -- start start 2023-07-06T02:24:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff4539d1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff4539d1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff44c0cd180 0x7ff44c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff4539d1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff44c06dc10 0x7ff44c157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff4539d1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c06e110 con 0x7ff44c06dc10 2023-07-06T02:24:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff4539d1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c06e3f0 con 0x7ff44c0cd180 2023-07-06T02:24:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff4539d1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c06e6d0 con 0x7ff44c0705a0 2023-07-06T02:24:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff451f6e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff44c06dc10 0x7ff44c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff451f6e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff44c06dc10 0x7ff44c157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34796/0 (socket says 172.21.15.89:34796) 2023-07-06T02:24:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff451f6e700 1 -- 172.21.15.89:0/1722499466 learned_addr learned my addr 172.21.15.89:0/1722499466 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff45176d700 1 --2- 172.21.15.89:0/1722499466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.407+0000 7ff450f6c700 1 --2- 172.21.15.89:0/1722499466 >> v2:172.21.15.172:3300/0 conn(0x7ff44c0cd180 0x7ff44c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff451f6e700 1 -- 172.21.15.89:0/1722499466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 msgr2=0x7ff44c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff451f6e700 1 --2- 172.21.15.89:0/1722499466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff451f6e700 1 -- 172.21.15.89:0/1722499466 >> v2:172.21.15.172:3300/0 conn(0x7ff44c0cd180 msgr2=0x7ff44c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff451f6e700 1 --2- 172.21.15.89:0/1722499466 >> v2:172.21.15.172:3300/0 conn(0x7ff44c0cd180 0x7ff44c15a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff451f6e700 1 -- 172.21.15.89:0/1722499466 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c06e9b0 con 0x7ff44c06dc10 2023-07-06T02:24:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff451f6e700 1 --2- 172.21.15.89:0/1722499466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c157950 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff44c072290 tx=0x7ff44400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac5f54e49c3e78f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff443fff700 1 -- 172.21.15.89:0/1722499466 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400e070 con 0x7ff44c06dc10 2023-07-06T02:24:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff443fff700 1 -- 172.21.15.89:0/1722499466 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400b9c0 con 0x7ff44c06dc10 2023-07-06T02:24:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff443fff700 1 -- 172.21.15.89:0/1722499466 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400bd70 con 0x7ff44c06dc10 2023-07-06T02:24:14.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 -- 172.21.15.89:0/1722499466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 msgr2=0x7ff44c157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1722499466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c157950 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff44c072290 tx=0x7ff44400b330 comp rx=0 tx=0).stop 2023-07-06T02:24:14.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 -- 172.21.15.89:0/1722499466 shutdown_connections 2023-07-06T02:24:14.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1722499466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1722499466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c157950 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1722499466 >> v2:172.21.15.172:3300/0 conn(0x7ff44c0cd180 0x7ff44c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 -- 172.21.15.89:0/1722499466 >> 172.21.15.89:0/1722499466 conn(0x7ff44c0d0b00 msgr2=0x7ff44c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:14.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.408+0000 7ff4539d1700 1 -- 172.21.15.89:0/1722499466 shutdown_connections 2023-07-06T02:24:14.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.409+0000 7ff4539d1700 1 -- 172.21.15.89:0/1722499466 wait complete. 2023-07-06T02:24:14.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.410+0000 7ff4539d1700 1 Processor -- start 2023-07-06T02:24:14.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.410+0000 7ff4539d1700 1 -- start start 2023-07-06T02:24:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff4539d1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff4539d1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c1f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff4539d1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff44c0cd180 0x7ff44c0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff4539d1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c1f0ca0 con 0x7ff44c06dc10 2023-07-06T02:24:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff4539d1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0c94e0 con 0x7ff44c0cd180 2023-07-06T02:24:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff4539d1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0c97c0 con 0x7ff44c0705a0 2023-07-06T02:24:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c1587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34798/0 (socket says 172.21.15.89:34798) 2023-07-06T02:24:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 -- 172.21.15.89:0/1403644390 learned_addr learned my addr 172.21.15.89:0/1403644390 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff450f6c700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c1f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff451f6e700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff44c0cd180 0x7ff44c0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 -- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 msgr2=0x7ff44c1f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c1f0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 -- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff44c0cd180 msgr2=0x7ff44c0ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff44c0cd180 0x7ff44c0ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 -- 172.21.15.89:0/1403644390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444009b30 con 0x7ff44c06dc10 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff45176d700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c1587f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00e810 tx=0x7ff43c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff441ffb700 1 -- 172.21.15.89:0/1403644390 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c013610 con 0x7ff44c06dc10 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff441ffb700 1 -- 172.21.15.89:0/1403644390 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c013770 con 0x7ff44c06dc10 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c0c9aa0 con 0x7ff44c06dc10 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.411+0000 7ff441ffb700 1 -- 172.21.15.89:0/1403644390 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c013b00 con 0x7ff44c06dc10 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.412+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c1f1340 con 0x7ff44c06dc10 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.413+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c0ca0f0 con 0x7ff44c06dc10 2023-07-06T02:24:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.413+0000 7ff441ffb700 1 -- 172.21.15.89:0/1403644390 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff43c013c60 con 0x7ff44c06dc10 2023-07-06T02:24:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.413+0000 7ff441ffb700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff43006acb0 0x7ff43006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.414+0000 7ff441ffb700 1 -- 172.21.15.89:0/1403644390 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff43c089270 con 0x7ff44c06dc10 2023-07-06T02:24:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.414+0000 7ff450f6c700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff43006acb0 0x7ff43006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.414+0000 7ff450f6c700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff43006acb0 0x7ff43006d160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0b69c0 tx=0x7ff448008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.418+0000 7ff441ffb700 1 -- 172.21.15.89:0/1403644390 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff43c0541e0 con 0x7ff44c06dc10 2023-07-06T02:24:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.605+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 --> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff44c15df00 con 0x7ff43006acb0 2023-07-06T02:24:14.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.614+0000 7ff441ffb700 1 -- 172.21.15.89:0/1403644390 <== mgr.4100 v2:172.21.15.89:6832/3591115711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311652 (secure 0 0 0) 0x7ff44800be50 con 0x7ff43006acb0 2023-07-06T02:24:14.620 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff43006acb0 msgr2=0x7ff43006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff43006acb0 0x7ff43006d160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0b69c0 tx=0x7ff448008040 comp rx=0 tx=0).stop 2023-07-06T02:24:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 msgr2=0x7ff44c1587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c1587f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00e810 tx=0x7ff43c00ebd0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 shutdown_connections 2023-07-06T02:24:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff44c0cd180 0x7ff44c0ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7ff43006acb0 0x7ff43006d160 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff44c06dc10 0x7ff44c1587f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 --2- 172.21.15.89:0/1403644390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff44c0705a0 0x7ff44c1f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 >> 172.21.15.89:0/1403644390 conn(0x7ff44c0d0b00 msgr2=0x7ff44c0b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 shutdown_connections 2023-07-06T02:24:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.617+0000 7ff4539d1700 1 -- 172.21.15.89:0/1403644390 wait complete. 2023-07-06T02:24:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-07-06T02:24:14.632 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-07-06T02:24:13.997937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220084,"kb_used_data":4468,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754636,"statfs":{"total":773094113280,"available":772868747264,"internally_reserved":0,"allocated":4575232,"data_stored":2677089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.494951"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555122+0000","last_change":"2023-07-06T02:24:10.547574+0000","last_active":"2023-07-06T02:24:10.555122+0000","last_peered":"2023-07-06T02:24:10.555122+0000","last_clean":"2023-07-06T02:24:10.555122+0000","last_became_active":"2023-07-06T02:24:10.547297+0000","last_became_peered":"2023-07-06T02:24:10.547297+0000","last_unstale":"2023-07-06T02:24:10.555122+0000","last_undegraded":"2023-07-06T02:24:10.555122+0000","last_fullsized":"2023-07-06T02:24:10.555122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:06:57.781915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554360+0000","last_change":"2023-07-06T02:24:10.550764+0000","last_active":"2023-07-06T02:24:10.554360+0000","last_peered":"2023-07-06T02:24:10.554360+0000","last_clean":"2023-07-06T02:24:10.554360+0000","last_became_active":"2023-07-06T02:24:10.547108+0000","last_became_peered":"2023-07-06T02:24:10.547108+0000","last_unstale":"2023-07-06T02:24:10.554360+0000","last_undegraded":"2023-07-06T02:24:10.554360+0000","last_fullsized":"2023-07-06T02:24:10.554360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:46:49.223079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551930+0000","last_change":"2023-07-06T02:24:10.545423+0000","last_active":"2023-07-06T02:24:10.551930+0000","last_peered":"2023-07-06T02:24:10.551930+0000","last_clean":"2023-07-06T02:24:10.551930+0000","last_became_active":"2023-07-06T02:24:10.545106+0000","last_became_peered":"2023-07-06T02:24:10.545106+0000","last_unstale":"2023-07-06T02:24:10.551930+0000","last_undegraded":"2023-07-06T02:24:10.551930+0000","last_fullsized":"2023-07-06T02:24:10.551930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:41.253417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557018+0000","last_change":"2023-07-06T02:24:10.542953+0000","last_active":"2023-07-06T02:24:10.557018+0000","last_peered":"2023-07-06T02:24:10.557018+0000","last_clean":"2023-07-06T02:24:10.557018+0000","last_became_active":"2023-07-06T02:24:10.542695+0000","last_became_peered":"2023-07-06T02:24:10.542695+0000","last_unstale":"2023-07-06T02:24:10.557018+0000","last_undegraded":"2023-07-06T02:24:10.557018+0000","last_fullsized":"2023-07-06T02:24:10.557018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:49:19.263018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554233+0000","last_change":"2023-07-06T02:24:10.546797+0000","last_active":"2023-07-06T02:24:10.554233+0000","last_peered":"2023-07-06T02:24:10.554233+0000","last_clean":"2023-07-06T02:24:10.554233+0000","last_became_active":"2023-07-06T02:24:10.546539+0000","last_became_peered":"2023-07-06T02:24:10.546539+0000","last_unstale":"2023-07-06T02:24:10.554233+0000","last_undegraded":"2023-07-06T02:24:10.554233+0000","last_fullsized":"2023-07-06T02:24:10.554233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:53.708345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.558843+0000","last_change":"2023-07-06T02:24:10.548795+0000","last_active":"2023-07-06T02:24:10.558843+0000","last_peered":"2023-07-06T02:24:10.558843+0000","last_clean":"2023-07-06T02:24:10.558843+0000","last_became_active":"2023-07-06T02:24:10.548511+0000","last_became_peered":"2023-07-06T02:24:10.548511+0000","last_unstale":"2023-07-06T02:24:10.558843+0000","last_undegraded":"2023-07-06T02:24:10.558843+0000","last_fullsized":"2023-07-06T02:24:10.558843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:54.087952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.548247+0000","last_change":"2023-07-06T02:24:10.544377+0000","last_active":"2023-07-06T02:24:10.548247+0000","last_peered":"2023-07-06T02:24:10.548247+0000","last_clean":"2023-07-06T02:24:10.548247+0000","last_became_active":"2023-07-06T02:24:10.544130+0000","last_became_peered":"2023-07-06T02:24:10.544130+0000","last_unstale":"2023-07-06T02:24:10.548247+0000","last_undegraded":"2023-07-06T02:24:10.548247+0000","last_fullsized":"2023-07-06T02:24:10.548247+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:29.993759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557534+0000","last_change":"2023-07-06T02:24:10.547933+0000","last_active":"2023-07-06T02:24:10.557534+0000","last_peered":"2023-07-06T02:24:10.557534+0000","last_clean":"2023-07-06T02:24:10.557534+0000","last_became_active":"2023-07-06T02:24:10.547360+0000","last_became_peered":"2023-07-06T02:24:10.547360+0000","last_unstale":"2023-07-06T02:24:10.557534+0000","last_undegraded":"2023-07-06T02:24:10.557534+0000","last_fullsized":"2023-07-06T02:24:10.557534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:33:16.750479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560451+0000","last_change":"2023-07-06T02:24:10.553189+0000","last_active":"2023-07-06T02:24:10.560451+0000","last_peered":"2023-07-06T02:24:10.560451+0000","last_clean":"2023-07-06T02:24:10.560451+0000","last_became_active":"2023-07-06T02:24:10.552894+0000","last_became_peered":"2023-07-06T02:24:10.552894+0000","last_unstale":"2023-07-06T02:24:10.560451+0000","last_undegraded":"2023-07-06T02:24:10.560451+0000","last_fullsized":"2023-07-06T02:24:10.560451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:02.469016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557210+0000","last_change":"2023-07-06T02:24:10.546568+0000","last_active":"2023-07-06T02:24:10.557210+0000","last_peered":"2023-07-06T02:24:10.557210+0000","last_clean":"2023-07-06T02:24:10.557210+0000","last_became_active":"2023-07-06T02:24:10.546114+0000","last_became_peered":"2023-07-06T02:24:10.546114+0000","last_unstale":"2023-07-06T02:24:10.557210+0000","last_undegraded":"2023-07-06T02:24:10.557210+0000","last_fullsized":"2023-07-06T02:24:10.557210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:08.271075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560620+0000","last_change":"2023-07-06T02:24:10.553581+0000","last_active":"2023-07-06T02:24:10.560620+0000","last_peered":"2023-07-06T02:24:10.560620+0000","last_clean":"2023-07-06T02:24:10.560620+0000","last_became_active":"2023-07-06T02:24:10.553310+0000","last_became_peered":"2023-07-06T02:24:10.553310+0000","last_unstale":"2023-07-06T02:24:10.560620+0000","last_undegraded":"2023-07-06T02:24:10.560620+0000","last_fullsized":"2023-07-06T02:24:10.560620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:58:24.057393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541810+0000","last_change":"2023-07-06T02:24:10.537491+0000","last_active":"2023-07-06T02:24:10.541810+0000","last_peered":"2023-07-06T02:24:10.541810+0000","last_clean":"2023-07-06T02:24:10.541810+0000","last_became_active":"2023-07-06T02:24:10.536814+0000","last_became_peered":"2023-07-06T02:24:10.536814+0000","last_unstale":"2023-07-06T02:24:10.541810+0000","last_undegraded":"2023-07-06T02:24:10.541810+0000","last_fullsized":"2023-07-06T02:24:10.541810+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:11.003933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560432+0000","last_change":"2023-07-06T02:24:10.557878+0000","last_active":"2023-07-06T02:24:10.560432+0000","last_peered":"2023-07-06T02:24:10.560432+0000","last_clean":"2023-07-06T02:24:10.560432+0000","last_became_active":"2023-07-06T02:24:10.557600+0000","last_became_peered":"2023-07-06T02:24:10.557600+0000","last_unstale":"2023-07-06T02:24:10.560432+0000","last_undegraded":"2023-07-06T02:24:10.560432+0000","last_fullsized":"2023-07-06T02:24:10.560432+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:04:51.093024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559572+0000","last_change":"2023-07-06T02:24:10.550524+0000","last_active":"2023-07-06T02:24:10.559572+0000","last_peered":"2023-07-06T02:24:10.559572+0000","last_clean":"2023-07-06T02:24:10.559572+0000","last_became_active":"2023-07-06T02:24:10.550255+0000","last_became_peered":"2023-07-06T02:24:10.550255+0000","last_unstale":"2023-07-06T02:24:10.559572+0000","last_undegraded":"2023-07-06T02:24:10.559572+0000","last_fullsized":"2023-07-06T02:24:10.559572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:39:34.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.558319+0000","last_change":"2023-07-06T02:24:10.550141+0000","last_active":"2023-07-06T02:24:10.558319+0000","last_peered":"2023-07-06T02:24:10.558319+0000","last_clean":"2023-07-06T02:24:10.558319+0000","last_became_active":"2023-07-06T02:24:10.549877+0000","last_became_peered":"2023-07-06T02:24:10.549877+0000","last_unstale":"2023-07-06T02:24:10.558319+0000","last_undegraded":"2023-07-06T02:24:10.558319+0000","last_fullsized":"2023-07-06T02:24:10.558319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:03.761474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555823+0000","last_change":"2023-07-06T02:24:10.553912+0000","last_active":"2023-07-06T02:24:10.555823+0000","last_peered":"2023-07-06T02:24:10.555823+0000","last_clean":"2023-07-06T02:24:10.555823+0000","last_became_active":"2023-07-06T02:24:10.553558+0000","last_became_peered":"2023-07-06T02:24:10.553558+0000","last_unstale":"2023-07-06T02:24:10.555823+0000","last_undegraded":"2023-07-06T02:24:10.555823+0000","last_fullsized":"2023-07-06T02:24:10.555823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:52:08.212360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551959+0000","last_change":"2023-07-06T02:24:10.546095+0000","last_active":"2023-07-06T02:24:10.551959+0000","last_peered":"2023-07-06T02:24:10.551959+0000","last_clean":"2023-07-06T02:24:10.551959+0000","last_became_active":"2023-07-06T02:24:10.544582+0000","last_became_peered":"2023-07-06T02:24:10.544582+0000","last_unstale":"2023-07-06T02:24:10.551959+0000","last_undegraded":"2023-07-06T02:24:10.551959+0000","last_fullsized":"2023-07-06T02:24:10.551959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:55:37.425912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554657+0000","last_change":"2023-07-06T02:24:10.547888+0000","last_active":"2023-07-06T02:24:10.554657+0000","last_peered":"2023-07-06T02:24:10.554657+0000","last_clean":"2023-07-06T02:24:10.554657+0000","last_became_active":"2023-07-06T02:24:10.545860+0000","last_became_peered":"2023-07-06T02:24:10.545860+0000","last_unstale":"2023-07-06T02:24:10.554657+0000","last_undegraded":"2023-07-06T02:24:10.554657+0000","last_fullsized":"2023-07-06T02:24:10.554657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:24.271903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.566255+0000","last_change":"2023-07-06T02:24:10.557841+0000","last_active":"2023-07-06T02:24:10.566255+0000","last_peered":"2023-07-06T02:24:10.566255+0000","last_clean":"2023-07-06T02:24:10.566255+0000","last_became_active":"2023-07-06T02:24:10.557614+0000","last_became_peered":"2023-07-06T02:24:10.557614+0000","last_unstale":"2023-07-06T02:24:10.566255+0000","last_undegraded":"2023-07-06T02:24:10.566255+0000","last_fullsized":"2023-07-06T02:24:10.566255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:23:11.474183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551568+0000","last_change":"2023-07-06T02:24:10.548573+0000","last_active":"2023-07-06T02:24:10.551568+0000","last_peered":"2023-07-06T02:24:10.551568+0000","last_clean":"2023-07-06T02:24:10.551568+0000","last_became_active":"2023-07-06T02:24:10.547705+0000","last_became_peered":"2023-07-06T02:24:10.547705+0000","last_unstale":"2023-07-06T02:24:10.551568+0000","last_undegraded":"2023-07-06T02:24:10.551568+0000","last_fullsized":"2023-07-06T02:24:10.551568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:54:52.631910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561772+0000","last_change":"2023-07-06T02:24:10.555704+0000","last_active":"2023-07-06T02:24:10.561772+0000","last_peered":"2023-07-06T02:24:10.561772+0000","last_clean":"2023-07-06T02:24:10.561772+0000","last_became_active":"2023-07-06T02:24:10.555555+0000","last_became_peered":"2023-07-06T02:24:10.555555+0000","last_unstale":"2023-07-06T02:24:10.561772+0000","last_undegraded":"2023-07-06T02:24:10.561772+0000","last_fullsized":"2023-07-06T02:24:10.561772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:21:40.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.556908+0000","last_change":"2023-07-06T02:24:10.544803+0000","last_active":"2023-07-06T02:24:10.556908+0000","last_peered":"2023-07-06T02:24:10.556908+0000","last_clean":"2023-07-06T02:24:10.556908+0000","last_became_active":"2023-07-06T02:24:10.544478+0000","last_became_peered":"2023-07-06T02:24:10.544478+0000","last_unstale":"2023-07-06T02:24:10.556908+0000","last_undegraded":"2023-07-06T02:24:10.556908+0000","last_fullsized":"2023-07-06T02:24:10.556908+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:43.358611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555015+0000","last_change":"2023-07-06T02:24:10.549075+0000","last_active":"2023-07-06T02:24:10.555015+0000","last_peered":"2023-07-06T02:24:10.555015+0000","last_clean":"2023-07-06T02:24:10.555015+0000","last_became_active":"2023-07-06T02:24:10.548634+0000","last_became_peered":"2023-07-06T02:24:10.548634+0000","last_unstale":"2023-07-06T02:24:10.555015+0000","last_undegraded":"2023-07-06T02:24:10.555015+0000","last_fullsized":"2023-07-06T02:24:10.555015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:58.637874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554901+0000","last_change":"2023-07-06T02:24:10.545014+0000","last_active":"2023-07-06T02:24:10.554901+0000","last_peered":"2023-07-06T02:24:10.554901+0000","last_clean":"2023-07-06T02:24:10.554901+0000","last_became_active":"2023-07-06T02:24:10.544701+0000","last_became_peered":"2023-07-06T02:24:10.544701+0000","last_unstale":"2023-07-06T02:24:10.554901+0000","last_undegraded":"2023-07-06T02:24:10.554901+0000","last_fullsized":"2023-07-06T02:24:10.554901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:40:00.950400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554849+0000","last_change":"2023-07-06T02:24:10.546635+0000","last_active":"2023-07-06T02:24:10.554849+0000","last_peered":"2023-07-06T02:24:10.554849+0000","last_clean":"2023-07-06T02:24:10.554849+0000","last_became_active":"2023-07-06T02:24:10.546370+0000","last_became_peered":"2023-07-06T02:24:10.546370+0000","last_unstale":"2023-07-06T02:24:10.554849+0000","last_undegraded":"2023-07-06T02:24:10.554849+0000","last_fullsized":"2023-07-06T02:24:10.554849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:50:45.017398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554732+0000","last_change":"2023-07-06T02:24:10.550324+0000","last_active":"2023-07-06T02:24:10.554732+0000","last_peered":"2023-07-06T02:24:10.554732+0000","last_clean":"2023-07-06T02:24:10.554732+0000","last_became_active":"2023-07-06T02:24:10.550042+0000","last_became_peered":"2023-07-06T02:24:10.550042+0000","last_unstale":"2023-07-06T02:24:10.554732+0000","last_undegraded":"2023-07-06T02:24:10.554732+0000","last_fullsized":"2023-07-06T02:24:10.554732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:29.095011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551903+0000","last_change":"2023-07-06T02:24:10.543676+0000","last_active":"2023-07-06T02:24:10.551903+0000","last_peered":"2023-07-06T02:24:10.551903+0000","last_clean":"2023-07-06T02:24:10.551903+0000","last_became_active":"2023-07-06T02:24:10.543400+0000","last_became_peered":"2023-07-06T02:24:10.543400+0000","last_unstale":"2023-07-06T02:24:10.551903+0000","last_undegraded":"2023-07-06T02:24:10.551903+0000","last_fullsized":"2023-07-06T02:24:10.551903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:26.079979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.542343+0000","last_change":"2023-07-06T02:24:10.537667+0000","last_active":"2023-07-06T02:24:10.542343+0000","last_peered":"2023-07-06T02:24:10.542343+0000","last_clean":"2023-07-06T02:24:10.542343+0000","last_became_active":"2023-07-06T02:24:10.537203+0000","last_became_peered":"2023-07-06T02:24:10.537203+0000","last_unstale":"2023-07-06T02:24:10.542343+0000","last_undegraded":"2023-07-06T02:24:10.542343+0000","last_fullsized":"2023-07-06T02:24:10.542343+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:35:42.843589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.553030+0000","last_change":"2023-07-06T02:24:10.545904+0000","last_active":"2023-07-06T02:24:10.553030+0000","last_peered":"2023-07-06T02:24:10.553030+0000","last_clean":"2023-07-06T02:24:10.553030+0000","last_became_active":"2023-07-06T02:24:10.545652+0000","last_became_peered":"2023-07-06T02:24:10.545652+0000","last_unstale":"2023-07-06T02:24:10.553030+0000","last_undegraded":"2023-07-06T02:24:10.553030+0000","last_fullsized":"2023-07-06T02:24:10.553030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:03:43.429442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551991+0000","last_change":"2023-07-06T02:24:10.543645+0000","last_active":"2023-07-06T02:24:10.551991+0000","last_peered":"2023-07-06T02:24:10.551991+0000","last_clean":"2023-07-06T02:24:10.551991+0000","last_became_active":"2023-07-06T02:24:10.543355+0000","last_became_peered":"2023-07-06T02:24:10.543355+0000","last_unstale":"2023-07-06T02:24:10.551991+0000","last_undegraded":"2023-07-06T02:24:10.551991+0000","last_fullsized":"2023-07-06T02:24:10.551991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:35.243003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561833+0000","last_change":"2023-07-06T02:24:10.558542+0000","last_active":"2023-07-06T02:24:10.561833+0000","last_peered":"2023-07-06T02:24:10.561833+0000","last_clean":"2023-07-06T02:24:10.561833+0000","last_became_active":"2023-07-06T02:24:10.558247+0000","last_became_peered":"2023-07-06T02:24:10.558247+0000","last_unstale":"2023-07-06T02:24:10.561833+0000","last_undegraded":"2023-07-06T02:24:10.561833+0000","last_fullsized":"2023-07-06T02:24:10.561833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:14.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.549350+0000","last_change":"2023-07-06T02:24:10.545571+0000","last_active":"2023-07-06T02:24:10.549350+0000","last_peered":"2023-07-06T02:24:10.549350+0000","last_clean":"2023-07-06T02:24:10.549350+0000","last_became_active":"2023-07-06T02:24:10.545274+0000","last_became_peered":"2023-07-06T02:24:10.545274+0000","last_unstale":"2023-07-06T02:24:10.549350+0000","last_undegraded":"2023-07-06T02:24:10.549350+0000","last_fullsized":"2023-07-06T02:24:10.549350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:01.368463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554097+0000","last_change":"2023-07-06T02:24:10.551064+0000","last_active":"2023-07-06T02:24:10.554097+0000","last_peered":"2023-07-06T02:24:10.554097+0000","last_clean":"2023-07-06T02:24:10.554097+0000","last_became_active":"2023-07-06T02:24:10.550787+0000","last_became_peered":"2023-07-06T02:24:10.550787+0000","last_unstale":"2023-07-06T02:24:10.554097+0000","last_undegraded":"2023-07-06T02:24:10.554097+0000","last_fullsized":"2023-07-06T02:24:10.554097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:35:47.914273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559239+0000","last_change":"2023-07-06T02:24:09.524883+0000","last_active":"2023-07-06T02:24:10.559239+0000","last_peered":"2023-07-06T02:24:10.559239+0000","last_clean":"2023-07-06T02:24:10.559239+0000","last_became_active":"2023-07-06T02:24:09.524444+0000","last_became_peered":"2023-07-06T02:24:09.524444+0000","last_unstale":"2023-07-06T02:24:10.559239+0000","last_undegraded":"2023-07-06T02:24:10.559239+0000","last_fullsized":"2023-07-06T02:24:10.559239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:15:29.718817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554935+0000","last_change":"2023-07-06T02:24:10.547891+0000","last_active":"2023-07-06T02:24:10.554935+0000","last_peered":"2023-07-06T02:24:10.554935+0000","last_clean":"2023-07-06T02:24:10.554935+0000","last_became_active":"2023-07-06T02:24:10.547595+0000","last_became_peered":"2023-07-06T02:24:10.547595+0000","last_unstale":"2023-07-06T02:24:10.554935+0000","last_undegraded":"2023-07-06T02:24:10.554935+0000","last_fullsized":"2023-07-06T02:24:10.554935+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:02:30.508728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.548005+0000","last_change":"2023-07-06T02:24:09.526186+0000","last_active":"2023-07-06T02:24:10.548005+0000","last_peered":"2023-07-06T02:24:10.548005+0000","last_clean":"2023-07-06T02:24:10.548005+0000","last_became_active":"2023-07-06T02:24:09.525901+0000","last_became_peered":"2023-07-06T02:24:09.525901+0000","last_unstale":"2023-07-06T02:24:10.548005+0000","last_undegraded":"2023-07-06T02:24:10.548005+0000","last_fullsized":"2023-07-06T02:24:10.548005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:17.022973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554982+0000","last_change":"2023-07-06T02:24:10.552868+0000","last_active":"2023-07-06T02:24:10.554982+0000","last_peered":"2023-07-06T02:24:10.554982+0000","last_clean":"2023-07-06T02:24:10.554982+0000","last_became_active":"2023-07-06T02:24:10.552616+0000","last_became_peered":"2023-07-06T02:24:10.552616+0000","last_unstale":"2023-07-06T02:24:10.554982+0000","last_undegraded":"2023-07-06T02:24:10.554982+0000","last_fullsized":"2023-07-06T02:24:10.554982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:44.486164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551941+0000","last_change":"2023-07-06T02:24:09.525143+0000","last_active":"2023-07-06T02:24:10.551941+0000","last_peered":"2023-07-06T02:24:10.551941+0000","last_clean":"2023-07-06T02:24:10.551941+0000","last_became_active":"2023-07-06T02:24:09.523406+0000","last_became_peered":"2023-07-06T02:24:09.523406+0000","last_unstale":"2023-07-06T02:24:10.551941+0000","last_undegraded":"2023-07-06T02:24:10.551941+0000","last_fullsized":"2023-07-06T02:24:10.551941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:15.941505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554137+0000","last_change":"2023-07-06T02:24:10.551787+0000","last_active":"2023-07-06T02:24:10.554137+0000","last_peered":"2023-07-06T02:24:10.554137+0000","last_clean":"2023-07-06T02:24:10.554137+0000","last_became_active":"2023-07-06T02:24:10.551418+0000","last_became_peered":"2023-07-06T02:24:10.551418+0000","last_unstale":"2023-07-06T02:24:10.554137+0000","last_undegraded":"2023-07-06T02:24:10.554137+0000","last_fullsized":"2023-07-06T02:24:10.554137+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:52:58.133143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547159+0000","last_change":"2023-07-06T02:24:09.528002+0000","last_active":"2023-07-06T02:24:10.547159+0000","last_peered":"2023-07-06T02:24:10.547159+0000","last_clean":"2023-07-06T02:24:10.547159+0000","last_became_active":"2023-07-06T02:24:09.527699+0000","last_became_peered":"2023-07-06T02:24:09.527699+0000","last_unstale":"2023-07-06T02:24:10.547159+0000","last_undegraded":"2023-07-06T02:24:10.547159+0000","last_fullsized":"2023-07-06T02:24:10.547159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:34:02.328706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.552010+0000","last_change":"2023-07-06T02:24:10.545453+0000","last_active":"2023-07-06T02:24:10.552010+0000","last_peered":"2023-07-06T02:24:10.552010+0000","last_clean":"2023-07-06T02:24:10.552010+0000","last_became_active":"2023-07-06T02:24:10.545090+0000","last_became_peered":"2023-07-06T02:24:10.545090+0000","last_unstale":"2023-07-06T02:24:10.552010+0000","last_undegraded":"2023-07-06T02:24:10.552010+0000","last_fullsized":"2023-07-06T02:24:10.552010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:29:51.253224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560675+0000","last_change":"2023-07-06T02:24:09.522080+0000","last_active":"2023-07-06T02:24:10.560675+0000","last_peered":"2023-07-06T02:24:10.560675+0000","last_clean":"2023-07-06T02:24:10.560675+0000","last_became_active":"2023-07-06T02:24:09.521312+0000","last_became_peered":"2023-07-06T02:24:09.521312+0000","last_unstale":"2023-07-06T02:24:10.560675+0000","last_undegraded":"2023-07-06T02:24:10.560675+0000","last_fullsized":"2023-07-06T02:24:10.560675+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:25.475842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559208+0000","last_change":"2023-07-06T02:24:10.548867+0000","last_active":"2023-07-06T02:24:10.559208+0000","last_peered":"2023-07-06T02:24:10.559208+0000","last_clean":"2023-07-06T02:24:10.559208+0000","last_became_active":"2023-07-06T02:24:10.548427+0000","last_became_peered":"2023-07-06T02:24:10.548427+0000","last_unstale":"2023-07-06T02:24:10.559208+0000","last_undegraded":"2023-07-06T02:24:10.559208+0000","last_fullsized":"2023-07-06T02:24:10.559208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:18:22.869326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547350+0000","last_change":"2023-07-06T02:24:09.523878+0000","last_active":"2023-07-06T02:24:10.547350+0000","last_peered":"2023-07-06T02:24:10.547350+0000","last_clean":"2023-07-06T02:24:10.547350+0000","last_became_active":"2023-07-06T02:24:09.523621+0000","last_became_peered":"2023-07-06T02:24:09.523621+0000","last_unstale":"2023-07-06T02:24:10.547350+0000","last_undegraded":"2023-07-06T02:24:10.547350+0000","last_fullsized":"2023-07-06T02:24:10.547350+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:17.177698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557553+0000","last_change":"2023-07-06T02:24:10.548079+0000","last_active":"2023-07-06T02:24:10.557553+0000","last_peered":"2023-07-06T02:24:10.557553+0000","last_clean":"2023-07-06T02:24:10.557553+0000","last_became_active":"2023-07-06T02:24:10.547630+0000","last_became_peered":"2023-07-06T02:24:10.547630+0000","last_unstale":"2023-07-06T02:24:10.557553+0000","last_undegraded":"2023-07-06T02:24:10.557553+0000","last_fullsized":"2023-07-06T02:24:10.557553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:13.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.526573+0000","last_change":"2023-07-06T02:24:09.523476+0000","last_active":"2023-07-06T02:24:10.526573+0000","last_peered":"2023-07-06T02:24:10.526573+0000","last_clean":"2023-07-06T02:24:10.526573+0000","last_became_active":"2023-07-06T02:24:09.523172+0000","last_became_peered":"2023-07-06T02:24:09.523172+0000","last_unstale":"2023-07-06T02:24:10.526573+0000","last_undegraded":"2023-07-06T02:24:10.526573+0000","last_fullsized":"2023-07-06T02:24:10.526573+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:38:10.219049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555034+0000","last_change":"2023-07-06T02:24:10.549234+0000","last_active":"2023-07-06T02:24:10.555034+0000","last_peered":"2023-07-06T02:24:10.555034+0000","last_clean":"2023-07-06T02:24:10.555034+0000","last_became_active":"2023-07-06T02:24:10.548920+0000","last_became_peered":"2023-07-06T02:24:10.548920+0000","last_unstale":"2023-07-06T02:24:10.555034+0000","last_undegraded":"2023-07-06T02:24:10.555034+0000","last_fullsized":"2023-07-06T02:24:10.555034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:17:14.239545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541311+0000","last_change":"2023-07-06T02:24:09.528575+0000","last_active":"2023-07-06T02:24:10.541311+0000","last_peered":"2023-07-06T02:24:10.541311+0000","last_clean":"2023-07-06T02:24:10.541311+0000","last_became_active":"2023-07-06T02:24:09.528028+0000","last_became_peered":"2023-07-06T02:24:09.528028+0000","last_unstale":"2023-07-06T02:24:10.541311+0000","last_undegraded":"2023-07-06T02:24:10.541311+0000","last_fullsized":"2023-07-06T02:24:10.541311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:38:05.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557641+0000","last_change":"2023-07-06T02:24:10.553773+0000","last_active":"2023-07-06T02:24:10.557641+0000","last_peered":"2023-07-06T02:24:10.557641+0000","last_clean":"2023-07-06T02:24:10.557641+0000","last_became_active":"2023-07-06T02:24:10.553240+0000","last_became_peered":"2023-07-06T02:24:10.553240+0000","last_unstale":"2023-07-06T02:24:10.557641+0000","last_undegraded":"2023-07-06T02:24:10.557641+0000","last_fullsized":"2023-07-06T02:24:10.557641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:21:47.220791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530847+0000","last_change":"2023-07-06T02:24:09.522755+0000","last_active":"2023-07-06T02:24:10.530847+0000","last_peered":"2023-07-06T02:24:10.530847+0000","last_clean":"2023-07-06T02:24:10.530847+0000","last_became_active":"2023-07-06T02:24:09.522270+0000","last_became_peered":"2023-07-06T02:24:09.522270+0000","last_unstale":"2023-07-06T02:24:10.530847+0000","last_undegraded":"2023-07-06T02:24:10.530847+0000","last_fullsized":"2023-07-06T02:24:10.530847+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:23:13.688739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.543043+0000","last_change":"2023-07-06T02:24:10.539321+0000","last_active":"2023-07-06T02:24:10.543043+0000","last_peered":"2023-07-06T02:24:10.543043+0000","last_clean":"2023-07-06T02:24:10.543043+0000","last_became_active":"2023-07-06T02:24:10.539066+0000","last_became_peered":"2023-07-06T02:24:10.539066+0000","last_unstale":"2023-07-06T02:24:10.543043+0000","last_undegraded":"2023-07-06T02:24:10.543043+0000","last_fullsized":"2023-07-06T02:24:10.543043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:21:40.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.543621+0000","last_change":"2023-07-06T02:24:09.530485+0000","last_active":"2023-07-06T02:24:10.543621+0000","last_peered":"2023-07-06T02:24:10.543621+0000","last_clean":"2023-07-06T02:24:10.543621+0000","last_became_active":"2023-07-06T02:24:09.530184+0000","last_became_peered":"2023-07-06T02:24:09.530184+0000","last_unstale":"2023-07-06T02:24:10.543621+0000","last_undegraded":"2023-07-06T02:24:10.543621+0000","last_fullsized":"2023-07-06T02:24:10.543621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:31:06.867675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561970+0000","last_change":"2023-07-06T02:24:10.559022+0000","last_active":"2023-07-06T02:24:10.561970+0000","last_peered":"2023-07-06T02:24:10.561970+0000","last_clean":"2023-07-06T02:24:10.561970+0000","last_became_active":"2023-07-06T02:24:10.558784+0000","last_became_peered":"2023-07-06T02:24:10.558784+0000","last_unstale":"2023-07-06T02:24:10.561970+0000","last_undegraded":"2023-07-06T02:24:10.561970+0000","last_fullsized":"2023-07-06T02:24:10.561970+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:37:08.277626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547366+0000","last_change":"2023-07-06T02:24:09.531056+0000","last_active":"2023-07-06T02:24:10.547366+0000","last_peered":"2023-07-06T02:24:10.547366+0000","last_clean":"2023-07-06T02:24:10.547366+0000","last_became_active":"2023-07-06T02:24:09.530682+0000","last_became_peered":"2023-07-06T02:24:09.530682+0000","last_unstale":"2023-07-06T02:24:10.547366+0000","last_undegraded":"2023-07-06T02:24:10.547366+0000","last_fullsized":"2023-07-06T02:24:10.547366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:37:07.271615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555398+0000","last_change":"2023-07-06T02:24:10.552688+0000","last_active":"2023-07-06T02:24:10.555398+0000","last_peered":"2023-07-06T02:24:10.555398+0000","last_clean":"2023-07-06T02:24:10.555398+0000","last_became_active":"2023-07-06T02:24:10.552418+0000","last_became_peered":"2023-07-06T02:24:10.552418+0000","last_unstale":"2023-07-06T02:24:10.555398+0000","last_undegraded":"2023-07-06T02:24:10.555398+0000","last_fullsized":"2023-07-06T02:24:10.555398+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:34.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.566834+0000","last_change":"2023-07-06T02:24:09.528086+0000","last_active":"2023-07-06T02:24:10.566834+0000","last_peered":"2023-07-06T02:24:10.566834+0000","last_clean":"2023-07-06T02:24:10.566834+0000","last_became_active":"2023-07-06T02:24:09.527768+0000","last_became_peered":"2023-07-06T02:24:09.527768+0000","last_unstale":"2023-07-06T02:24:10.566834+0000","last_undegraded":"2023-07-06T02:24:10.566834+0000","last_fullsized":"2023-07-06T02:24:10.566834+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:30:36.725924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554879+0000","last_change":"2023-07-06T02:24:10.546835+0000","last_active":"2023-07-06T02:24:10.554879+0000","last_peered":"2023-07-06T02:24:10.554879+0000","last_clean":"2023-07-06T02:24:10.554879+0000","last_became_active":"2023-07-06T02:24:10.546533+0000","last_became_peered":"2023-07-06T02:24:10.546533+0000","last_unstale":"2023-07-06T02:24:10.554879+0000","last_undegraded":"2023-07-06T02:24:10.554879+0000","last_fullsized":"2023-07-06T02:24:10.554879+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:57:37.805536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.563265+0000","last_change":"2023-07-06T02:24:09.529989+0000","last_active":"2023-07-06T02:24:10.563265+0000","last_peered":"2023-07-06T02:24:10.563265+0000","last_clean":"2023-07-06T02:24:10.563265+0000","last_became_active":"2023-07-06T02:24:09.529734+0000","last_became_peered":"2023-07-06T02:24:09.529734+0000","last_unstale":"2023-07-06T02:24:10.563265+0000","last_undegraded":"2023-07-06T02:24:10.563265+0000","last_fullsized":"2023-07-06T02:24:10.563265+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:48.336132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559189+0000","last_change":"2023-07-06T02:24:10.546105+0000","last_active":"2023-07-06T02:24:10.559189+0000","last_peered":"2023-07-06T02:24:10.559189+0000","last_clean":"2023-07-06T02:24:10.559189+0000","last_became_active":"2023-07-06T02:24:10.545831+0000","last_became_peered":"2023-07-06T02:24:10.545831+0000","last_unstale":"2023-07-06T02:24:10.559189+0000","last_undegraded":"2023-07-06T02:24:10.559189+0000","last_fullsized":"2023-07-06T02:24:10.559189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:53.030078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.529725+0000","last_change":"2023-07-06T02:24:09.527980+0000","last_active":"2023-07-06T02:24:10.529725+0000","last_peered":"2023-07-06T02:24:10.529725+0000","last_clean":"2023-07-06T02:24:10.529725+0000","last_became_active":"2023-07-06T02:24:09.527641+0000","last_became_peered":"2023-07-06T02:24:09.527641+0000","last_unstale":"2023-07-06T02:24:10.529725+0000","last_undegraded":"2023-07-06T02:24:10.529725+0000","last_fullsized":"2023-07-06T02:24:10.529725+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:31:06.867675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561755+0000","last_change":"2023-07-06T02:24:10.555431+0000","last_active":"2023-07-06T02:24:10.561755+0000","last_peered":"2023-07-06T02:24:10.561755+0000","last_clean":"2023-07-06T02:24:10.561755+0000","last_became_active":"2023-07-06T02:24:10.555108+0000","last_became_peered":"2023-07-06T02:24:10.555108+0000","last_unstale":"2023-07-06T02:24:10.561755+0000","last_undegraded":"2023-07-06T02:24:10.561755+0000","last_fullsized":"2023-07-06T02:24:10.561755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:29.778784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.539655+0000","last_change":"2023-07-06T02:24:09.531563+0000","last_active":"2023-07-06T02:24:10.539655+0000","last_peered":"2023-07-06T02:24:10.539655+0000","last_clean":"2023-07-06T02:24:10.539655+0000","last_became_active":"2023-07-06T02:24:09.531278+0000","last_became_peered":"2023-07-06T02:24:09.531278+0000","last_unstale":"2023-07-06T02:24:10.539655+0000","last_undegraded":"2023-07-06T02:24:10.539655+0000","last_fullsized":"2023-07-06T02:24:10.539655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:06:10.921079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.548236+0000","last_change":"2023-07-06T02:24:10.540350+0000","last_active":"2023-07-06T02:24:10.548236+0000","last_peered":"2023-07-06T02:24:10.548236+0000","last_clean":"2023-07-06T02:24:10.548236+0000","last_became_active":"2023-07-06T02:24:10.540079+0000","last_became_peered":"2023-07-06T02:24:10.540079+0000","last_unstale":"2023-07-06T02:24:10.548236+0000","last_undegraded":"2023-07-06T02:24:10.548236+0000","last_fullsized":"2023-07-06T02:24:10.548236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:14.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559856+0000","last_change":"2023-07-06T02:24:09.528422+0000","last_active":"2023-07-06T02:24:10.559856+0000","last_peered":"2023-07-06T02:24:10.559856+0000","last_clean":"2023-07-06T02:24:10.559856+0000","last_became_active":"2023-07-06T02:24:09.527703+0000","last_became_peered":"2023-07-06T02:24:09.527703+0000","last_unstale":"2023-07-06T02:24:10.559856+0000","last_undegraded":"2023-07-06T02:24:10.559856+0000","last_fullsized":"2023-07-06T02:24:10.559856+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:57:44.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.546950+0000","last_change":"2023-07-06T02:24:10.536109+0000","last_active":"2023-07-06T02:24:10.546950+0000","last_peered":"2023-07-06T02:24:10.546950+0000","last_clean":"2023-07-06T02:24:10.546950+0000","last_became_active":"2023-07-06T02:24:10.535837+0000","last_became_peered":"2023-07-06T02:24:10.535837+0000","last_unstale":"2023-07-06T02:24:10.546950+0000","last_undegraded":"2023-07-06T02:24:10.546950+0000","last_fullsized":"2023-07-06T02:24:10.546950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:47:39.164709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559910+0000","last_change":"2023-07-06T02:24:09.524241+0000","last_active":"2023-07-06T02:24:10.559910+0000","last_peered":"2023-07-06T02:24:10.559910+0000","last_clean":"2023-07-06T02:24:10.559910+0000","last_became_active":"2023-07-06T02:24:09.524006+0000","last_became_peered":"2023-07-06T02:24:09.524006+0000","last_unstale":"2023-07-06T02:24:10.559910+0000","last_undegraded":"2023-07-06T02:24:10.559910+0000","last_fullsized":"2023-07-06T02:24:10.559910+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:16.101315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559764+0000","last_change":"2023-07-06T02:24:10.552211+0000","last_active":"2023-07-06T02:24:10.559764+0000","last_peered":"2023-07-06T02:24:10.559764+0000","last_clean":"2023-07-06T02:24:10.559764+0000","last_became_active":"2023-07-06T02:24:10.551946+0000","last_became_peered":"2023-07-06T02:24:10.551946+0000","last_unstale":"2023-07-06T02:24:10.559764+0000","last_undegraded":"2023-07-06T02:24:10.559764+0000","last_fullsized":"2023-07-06T02:24:10.559764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:51:50.803477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.550023+0000","last_change":"2023-07-06T02:24:09.522250+0000","last_active":"2023-07-06T02:24:10.550023+0000","last_peered":"2023-07-06T02:24:10.550023+0000","last_clean":"2023-07-06T02:24:10.550023+0000","last_became_active":"2023-07-06T02:24:09.521482+0000","last_became_peered":"2023-07-06T02:24:09.521482+0000","last_unstale":"2023-07-06T02:24:10.550023+0000","last_undegraded":"2023-07-06T02:24:10.550023+0000","last_fullsized":"2023-07-06T02:24:10.550023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:06.553635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555845+0000","last_change":"2023-07-06T02:24:10.554357+0000","last_active":"2023-07-06T02:24:10.555845+0000","last_peered":"2023-07-06T02:24:10.555845+0000","last_clean":"2023-07-06T02:24:10.555845+0000","last_became_active":"2023-07-06T02:24:10.554046+0000","last_became_peered":"2023-07-06T02:24:10.554046+0000","last_unstale":"2023-07-06T02:24:10.555845+0000","last_undegraded":"2023-07-06T02:24:10.555845+0000","last_fullsized":"2023-07-06T02:24:10.555845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:29:26.733224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551785+0000","last_change":"2023-07-06T02:24:09.523544+0000","last_active":"2023-07-06T02:24:10.551785+0000","last_peered":"2023-07-06T02:24:10.551785+0000","last_clean":"2023-07-06T02:24:10.551785+0000","last_became_active":"2023-07-06T02:24:09.523242+0000","last_became_peered":"2023-07-06T02:24:09.523242+0000","last_unstale":"2023-07-06T02:24:10.551785+0000","last_undegraded":"2023-07-06T02:24:10.551785+0000","last_fullsized":"2023-07-06T02:24:10.551785+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:18:23.789530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555697+0000","last_change":"2023-07-06T02:24:10.552869+0000","last_active":"2023-07-06T02:24:10.555697+0000","last_peered":"2023-07-06T02:24:10.555697+0000","last_clean":"2023-07-06T02:24:10.555697+0000","last_became_active":"2023-07-06T02:24:10.552542+0000","last_became_peered":"2023-07-06T02:24:10.552542+0000","last_unstale":"2023-07-06T02:24:10.555697+0000","last_undegraded":"2023-07-06T02:24:10.555697+0000","last_fullsized":"2023-07-06T02:24:10.555697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:45.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.537819+0000","last_change":"2023-07-06T02:24:09.528988+0000","last_active":"2023-07-06T02:24:10.537819+0000","last_peered":"2023-07-06T02:24:10.537819+0000","last_clean":"2023-07-06T02:24:10.537819+0000","last_became_active":"2023-07-06T02:24:09.528698+0000","last_became_peered":"2023-07-06T02:24:09.528698+0000","last_unstale":"2023-07-06T02:24:10.537819+0000","last_undegraded":"2023-07-06T02:24:10.537819+0000","last_fullsized":"2023-07-06T02:24:10.537819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:38.533140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561846+0000","last_change":"2023-07-06T02:24:10.558634+0000","last_active":"2023-07-06T02:24:10.561846+0000","last_peered":"2023-07-06T02:24:10.561846+0000","last_clean":"2023-07-06T02:24:10.561846+0000","last_became_active":"2023-07-06T02:24:10.558377+0000","last_became_peered":"2023-07-06T02:24:10.558377+0000","last_unstale":"2023-07-06T02:24:10.561846+0000","last_undegraded":"2023-07-06T02:24:10.561846+0000","last_fullsized":"2023-07-06T02:24:10.561846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:34.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561486+0000","last_change":"2023-07-06T02:24:09.522239+0000","last_active":"2023-07-06T02:24:10.561486+0000","last_peered":"2023-07-06T02:24:10.561486+0000","last_clean":"2023-07-06T02:24:10.561486+0000","last_became_active":"2023-07-06T02:24:09.521822+0000","last_became_peered":"2023-07-06T02:24:09.521822+0000","last_unstale":"2023-07-06T02:24:10.561486+0000","last_undegraded":"2023-07-06T02:24:10.561486+0000","last_fullsized":"2023-07-06T02:24:10.561486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:24:37.602945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.546638+0000","last_change":"2023-07-06T02:24:10.543291+0000","last_active":"2023-07-06T02:24:10.546638+0000","last_peered":"2023-07-06T02:24:10.546638+0000","last_clean":"2023-07-06T02:24:10.546638+0000","last_became_active":"2023-07-06T02:24:10.543043+0000","last_became_peered":"2023-07-06T02:24:10.543043+0000","last_unstale":"2023-07-06T02:24:10.546638+0000","last_undegraded":"2023-07-06T02:24:10.546638+0000","last_fullsized":"2023-07-06T02:24:10.546638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:59:18.144786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.526259+0000","last_change":"2023-07-06T02:24:09.529211+0000","last_active":"2023-07-06T02:24:10.526259+0000","last_peered":"2023-07-06T02:24:10.526259+0000","last_clean":"2023-07-06T02:24:10.526259+0000","last_became_active":"2023-07-06T02:24:09.528945+0000","last_became_peered":"2023-07-06T02:24:09.528945+0000","last_unstale":"2023-07-06T02:24:10.526259+0000","last_undegraded":"2023-07-06T02:24:10.526259+0000","last_fullsized":"2023-07-06T02:24:10.526259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:32:52.988475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.545463+0000","last_change":"2023-07-06T02:24:10.537932+0000","last_active":"2023-07-06T02:24:10.545463+0000","last_peered":"2023-07-06T02:24:10.545463+0000","last_clean":"2023-07-06T02:24:10.545463+0000","last_became_active":"2023-07-06T02:24:10.537641+0000","last_became_peered":"2023-07-06T02:24:10.537641+0000","last_unstale":"2023-07-06T02:24:10.545463+0000","last_undegraded":"2023-07-06T02:24:10.545463+0000","last_fullsized":"2023-07-06T02:24:10.545463+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:55.365517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.543762+0000","last_change":"2023-07-06T02:24:09.530739+0000","last_active":"2023-07-06T02:24:10.543762+0000","last_peered":"2023-07-06T02:24:10.543762+0000","last_clean":"2023-07-06T02:24:10.543762+0000","last_became_active":"2023-07-06T02:24:09.530478+0000","last_became_peered":"2023-07-06T02:24:09.530478+0000","last_unstale":"2023-07-06T02:24:10.543762+0000","last_undegraded":"2023-07-06T02:24:10.543762+0000","last_fullsized":"2023-07-06T02:24:10.543762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:28.987747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557037+0000","last_change":"2023-07-06T02:24:10.545188+0000","last_active":"2023-07-06T02:24:10.557037+0000","last_peered":"2023-07-06T02:24:10.557037+0000","last_clean":"2023-07-06T02:24:10.557037+0000","last_became_active":"2023-07-06T02:24:10.544930+0000","last_became_peered":"2023-07-06T02:24:10.544930+0000","last_unstale":"2023-07-06T02:24:10.557037+0000","last_undegraded":"2023-07-06T02:24:10.557037+0000","last_fullsized":"2023-07-06T02:24:10.557037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:28.119966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530182+0000","last_change":"2023-07-06T02:24:09.527830+0000","last_active":"2023-07-06T02:24:10.530182+0000","last_peered":"2023-07-06T02:24:10.530182+0000","last_clean":"2023-07-06T02:24:10.530182+0000","last_became_active":"2023-07-06T02:24:09.527543+0000","last_became_peered":"2023-07-06T02:24:09.527543+0000","last_unstale":"2023-07-06T02:24:10.530182+0000","last_undegraded":"2023-07-06T02:24:10.530182+0000","last_fullsized":"2023-07-06T02:24:10.530182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:51.359531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.562038+0000","last_change":"2023-07-06T02:24:10.559205+0000","last_active":"2023-07-06T02:24:10.562038+0000","last_peered":"2023-07-06T02:24:10.562038+0000","last_clean":"2023-07-06T02:24:10.562038+0000","last_became_active":"2023-07-06T02:24:10.558876+0000","last_became_peered":"2023-07-06T02:24:10.558876+0000","last_unstale":"2023-07-06T02:24:10.562038+0000","last_undegraded":"2023-07-06T02:24:10.562038+0000","last_fullsized":"2023-07-06T02:24:10.562038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:12:59.248820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.531433+0000","last_change":"2023-07-06T02:24:09.522250+0000","last_active":"2023-07-06T02:24:10.531433+0000","last_peered":"2023-07-06T02:24:10.531433+0000","last_clean":"2023-07-06T02:24:10.531433+0000","last_became_active":"2023-07-06T02:24:09.521909+0000","last_became_peered":"2023-07-06T02:24:09.521909+0000","last_unstale":"2023-07-06T02:24:10.531433+0000","last_undegraded":"2023-07-06T02:24:10.531433+0000","last_fullsized":"2023-07-06T02:24:10.531433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:59.713337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530186+0000","last_change":"2023-07-06T02:22:47.973696+0000","last_active":"2023-07-06T02:24:10.530186+0000","last_peered":"2023-07-06T02:24:10.530186+0000","last_clean":"2023-07-06T02:24:10.530186+0000","last_became_active":"2023-07-06T02:22:47.973358+0000","last_became_peered":"2023-07-06T02:22:47.973358+0000","last_unstale":"2023-07-06T02:24:10.530186+0000","last_undegraded":"2023-07-06T02:24:10.530186+0000","last_fullsized":"2023-07-06T02:24:10.530186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:34:30.278326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.558866+0000","last_change":"2023-07-06T02:24:10.553865+0000","last_active":"2023-07-06T02:24:10.558866+0000","last_peered":"2023-07-06T02:24:10.558866+0000","last_clean":"2023-07-06T02:24:10.558866+0000","last_became_active":"2023-07-06T02:24:10.553526+0000","last_became_peered":"2023-07-06T02:24:10.553526+0000","last_unstale":"2023-07-06T02:24:10.558866+0000","last_undegraded":"2023-07-06T02:24:10.558866+0000","last_fullsized":"2023-07-06T02:24:10.558866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:06:57.781915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.531751+0000","last_change":"2023-07-06T02:24:09.526273+0000","last_active":"2023-07-06T02:24:10.531751+0000","last_peered":"2023-07-06T02:24:10.531751+0000","last_clean":"2023-07-06T02:24:10.531751+0000","last_became_active":"2023-07-06T02:24:09.525994+0000","last_became_peered":"2023-07-06T02:24:09.525994+0000","last_unstale":"2023-07-06T02:24:10.531751+0000","last_undegraded":"2023-07-06T02:24:10.531751+0000","last_fullsized":"2023-07-06T02:24:10.531751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:02.755463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541431+0000","last_change":"2023-07-06T02:22:47.323237+0000","last_active":"2023-07-06T02:24:10.541431+0000","last_peered":"2023-07-06T02:24:10.541431+0000","last_clean":"2023-07-06T02:24:10.541431+0000","last_became_active":"2023-07-06T02:22:47.322763+0000","last_became_peered":"2023-07-06T02:22:47.322763+0000","last_unstale":"2023-07-06T02:24:10.541431+0000","last_undegraded":"2023-07-06T02:24:10.541431+0000","last_fullsized":"2023-07-06T02:24:10.541431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:32:41.559971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547054+0000","last_change":"2023-07-06T02:24:10.545773+0000","last_active":"2023-07-06T02:24:10.547054+0000","last_peered":"2023-07-06T02:24:10.547054+0000","last_clean":"2023-07-06T02:24:10.547054+0000","last_became_active":"2023-07-06T02:24:10.545508+0000","last_became_peered":"2023-07-06T02:24:10.545508+0000","last_unstale":"2023-07-06T02:24:10.547054+0000","last_undegraded":"2023-07-06T02:24:10.547054+0000","last_fullsized":"2023-07-06T02:24:10.547054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:29.778784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.540615+0000","last_change":"2023-07-06T02:24:09.525690+0000","last_active":"2023-07-06T02:24:10.540615+0000","last_peered":"2023-07-06T02:24:10.540615+0000","last_clean":"2023-07-06T02:24:10.540615+0000","last_became_active":"2023-07-06T02:24:09.525499+0000","last_became_peered":"2023-07-06T02:24:09.525499+0000","last_unstale":"2023-07-06T02:24:10.540615+0000","last_undegraded":"2023-07-06T02:24:10.540615+0000","last_fullsized":"2023-07-06T02:24:10.540615+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:21:23.653397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.542212+0000","last_change":"2023-07-06T02:22:47.325598+0000","last_active":"2023-07-06T02:24:10.542212+0000","last_peered":"2023-07-06T02:24:10.542212+0000","last_clean":"2023-07-06T02:24:10.542212+0000","last_became_active":"2023-07-06T02:22:47.325315+0000","last_became_peered":"2023-07-06T02:22:47.325315+0000","last_unstale":"2023-07-06T02:24:10.542212+0000","last_undegraded":"2023-07-06T02:24:10.542212+0000","last_fullsized":"2023-07-06T02:24:10.542212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:50:03.777203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530438+0000","last_change":"2023-07-06T02:22:47.326882+0000","last_active":"2023-07-06T02:24:10.530438+0000","last_peered":"2023-07-06T02:24:10.530438+0000","last_clean":"2023-07-06T02:24:10.530438+0000","last_became_active":"2023-07-06T02:22:47.326511+0000","last_became_peered":"2023-07-06T02:22:47.326511+0000","last_unstale":"2023-07-06T02:24:10.530438+0000","last_undegraded":"2023-07-06T02:24:10.530438+0000","last_fullsized":"2023-07-06T02:24:10.530438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:52:30.894947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560488+0000","last_change":"2023-07-06T02:24:10.553509+0000","last_active":"2023-07-06T02:24:10.560488+0000","last_peered":"2023-07-06T02:24:10.560488+0000","last_clean":"2023-07-06T02:24:10.560488+0000","last_became_active":"2023-07-06T02:24:10.553222+0000","last_became_peered":"2023-07-06T02:24:10.553222+0000","last_unstale":"2023-07-06T02:24:10.560488+0000","last_undegraded":"2023-07-06T02:24:10.560488+0000","last_fullsized":"2023-07-06T02:24:10.560488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:23.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.533880+0000","last_change":"2023-07-06T02:24:09.528924+0000","last_active":"2023-07-06T02:24:10.533880+0000","last_peered":"2023-07-06T02:24:10.533880+0000","last_clean":"2023-07-06T02:24:10.533880+0000","last_became_active":"2023-07-06T02:24:09.528612+0000","last_became_peered":"2023-07-06T02:24:09.528612+0000","last_unstale":"2023-07-06T02:24:10.533880+0000","last_undegraded":"2023-07-06T02:24:10.533880+0000","last_fullsized":"2023-07-06T02:24:10.533880+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:16.122139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.529950+0000","last_change":"2023-07-06T02:22:47.613219+0000","last_active":"2023-07-06T02:24:10.529950+0000","last_peered":"2023-07-06T02:24:10.529950+0000","last_clean":"2023-07-06T02:24:10.529950+0000","last_became_active":"2023-07-06T02:22:47.612787+0000","last_became_peered":"2023-07-06T02:22:47.612787+0000","last_unstale":"2023-07-06T02:24:10.529950+0000","last_undegraded":"2023-07-06T02:24:10.529950+0000","last_fullsized":"2023-07-06T02:24:10.529950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:48:56.335387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557190+0000","last_change":"2023-07-06T02:24:10.545239+0000","last_active":"2023-07-06T02:24:10.557190+0000","last_peered":"2023-07-06T02:24:10.557190+0000","last_clean":"2023-07-06T02:24:10.557190+0000","last_became_active":"2023-07-06T02:24:10.544980+0000","last_became_peered":"2023-07-06T02:24:10.544980+0000","last_unstale":"2023-07-06T02:24:10.557190+0000","last_undegraded":"2023-07-06T02:24:10.557190+0000","last_fullsized":"2023-07-06T02:24:10.557190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:07:17.625774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559892+0000","last_change":"2023-07-06T02:24:09.524461+0000","last_active":"2023-07-06T02:24:10.559892+0000","last_peered":"2023-07-06T02:24:10.559892+0000","last_clean":"2023-07-06T02:24:10.559892+0000","last_became_active":"2023-07-06T02:24:09.524163+0000","last_became_peered":"2023-07-06T02:24:09.524163+0000","last_unstale":"2023-07-06T02:24:10.559892+0000","last_undegraded":"2023-07-06T02:24:10.559892+0000","last_fullsized":"2023-07-06T02:24:10.559892+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:37:12.190601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530394+0000","last_change":"2023-07-06T02:22:47.613102+0000","last_active":"2023-07-06T02:24:10.530394+0000","last_peered":"2023-07-06T02:24:10.530394+0000","last_clean":"2023-07-06T02:24:10.530394+0000","last_became_active":"2023-07-06T02:22:47.612655+0000","last_became_peered":"2023-07-06T02:22:47.612655+0000","last_unstale":"2023-07-06T02:24:10.530394+0000","last_undegraded":"2023-07-06T02:24:10.530394+0000","last_fullsized":"2023-07-06T02:24:10.530394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:11:36.055815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555792+0000","last_change":"2023-07-06T02:24:10.553589+0000","last_active":"2023-07-06T02:24:10.555792+0000","last_peered":"2023-07-06T02:24:10.555792+0000","last_clean":"2023-07-06T02:24:10.555792+0000","last_became_active":"2023-07-06T02:24:10.553269+0000","last_became_peered":"2023-07-06T02:24:10.553269+0000","last_unstale":"2023-07-06T02:24:10.555792+0000","last_undegraded":"2023-07-06T02:24:10.555792+0000","last_fullsized":"2023-07-06T02:24:10.555792+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:23:07.130763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.537795+0000","last_change":"2023-07-06T02:24:09.524266+0000","last_active":"2023-07-06T02:24:10.537795+0000","last_peered":"2023-07-06T02:24:10.537795+0000","last_clean":"2023-07-06T02:24:10.537795+0000","last_became_active":"2023-07-06T02:24:09.524003+0000","last_became_peered":"2023-07-06T02:24:09.524003+0000","last_unstale":"2023-07-06T02:24:10.537795+0000","last_undegraded":"2023-07-06T02:24:10.537795+0000","last_fullsized":"2023-07-06T02:24:10.537795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:16.122139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.526759+0000","last_change":"2023-07-06T02:22:47.613009+0000","last_active":"2023-07-06T02:24:10.526759+0000","last_peered":"2023-07-06T02:24:10.526759+0000","last_clean":"2023-07-06T02:24:10.526759+0000","last_became_active":"2023-07-06T02:22:47.612664+0000","last_became_peered":"2023-07-06T02:22:47.612664+0000","last_unstale":"2023-07-06T02:24:10.526759+0000","last_undegraded":"2023-07-06T02:24:10.526759+0000","last_fullsized":"2023-07-06T02:24:10.526759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:33:17.278717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555076+0000","last_change":"2023-07-06T02:24:10.552095+0000","last_active":"2023-07-06T02:24:10.555076+0000","last_peered":"2023-07-06T02:24:10.555076+0000","last_clean":"2023-07-06T02:24:10.555076+0000","last_became_active":"2023-07-06T02:24:10.551820+0000","last_became_peered":"2023-07-06T02:24:10.551820+0000","last_unstale":"2023-07-06T02:24:10.555076+0000","last_undegraded":"2023-07-06T02:24:10.555076+0000","last_fullsized":"2023-07-06T02:24:10.555076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:45.776434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541893+0000","last_change":"2023-07-06T02:24:09.528276+0000","last_active":"2023-07-06T02:24:10.541893+0000","last_peered":"2023-07-06T02:24:10.541893+0000","last_clean":"2023-07-06T02:24:10.541893+0000","last_became_active":"2023-07-06T02:24:09.527975+0000","last_became_peered":"2023-07-06T02:24:09.527975+0000","last_unstale":"2023-07-06T02:24:10.541893+0000","last_undegraded":"2023-07-06T02:24:10.541893+0000","last_fullsized":"2023-07-06T02:24:10.541893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:39:44.776219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.531349+0000","last_change":"2023-07-06T02:22:47.612949+0000","last_active":"2023-07-06T02:24:10.531349+0000","last_peered":"2023-07-06T02:24:10.531349+0000","last_clean":"2023-07-06T02:24:10.531349+0000","last_became_active":"2023-07-06T02:22:47.612606+0000","last_became_peered":"2023-07-06T02:22:47.612606+0000","last_unstale":"2023-07-06T02:24:10.531349+0000","last_undegraded":"2023-07-06T02:24:10.531349+0000","last_fullsized":"2023-07-06T02:24:10.531349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:38:29.300159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554722+0000","last_change":"2023-07-06T02:24:10.549495+0000","last_active":"2023-07-06T02:24:10.554722+0000","last_peered":"2023-07-06T02:24:10.554722+0000","last_clean":"2023-07-06T02:24:10.554722+0000","last_became_active":"2023-07-06T02:24:10.549257+0000","last_became_peered":"2023-07-06T02:24:10.549257+0000","last_unstale":"2023-07-06T02:24:10.554722+0000","last_undegraded":"2023-07-06T02:24:10.554722+0000","last_fullsized":"2023-07-06T02:24:10.554722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:43:57.009636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541602+0000","last_change":"2023-07-06T02:24:09.528277+0000","last_active":"2023-07-06T02:24:10.541602+0000","last_peered":"2023-07-06T02:24:10.541602+0000","last_clean":"2023-07-06T02:24:10.541602+0000","last_became_active":"2023-07-06T02:24:09.527375+0000","last_became_peered":"2023-07-06T02:24:09.527375+0000","last_unstale":"2023-07-06T02:24:10.541602+0000","last_undegraded":"2023-07-06T02:24:10.541602+0000","last_fullsized":"2023-07-06T02:24:10.541602+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:43.538847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.536610+0000","last_change":"2023-07-06T02:22:47.329340+0000","last_active":"2023-07-06T02:24:10.536610+0000","last_peered":"2023-07-06T02:24:10.536610+0000","last_clean":"2023-07-06T02:24:10.536610+0000","last_became_active":"2023-07-06T02:22:47.329090+0000","last_became_peered":"2023-07-06T02:22:47.329090+0000","last_unstale":"2023-07-06T02:24:10.536610+0000","last_undegraded":"2023-07-06T02:24:10.536610+0000","last_fullsized":"2023-07-06T02:24:10.536610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:43:10.456578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":1024000,"data_stored":781094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.501}]},{"osd":1,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.372},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.45600000000000002}]},{"osd":5,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":434176,"data_stored":191308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.623},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.435},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.46500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":1024000,"data_stored":781074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.375}]},{"osd":3,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.755}]},{"osd":6,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.76700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":409600,"data_stored":181827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.317},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.27200000000000002}]},{"osd":3,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.372},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.504},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.442},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":192222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.377}]},{"osd":3,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":183188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.255},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.317}]},{"osd":4,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.497}]},{"osd":7,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":183188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":183188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.623},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T02:24:14.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T02:24:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.790+0000 7f65ce43d700 1 Processor -- start 2023-07-06T02:24:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.790+0000 7f65ce43d700 1 -- start start 2023-07-06T02:24:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65ce43d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 0x7f65c80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65ce43d700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80cb3d0 con 0x7f65c8156ab0 2023-07-06T02:24:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65ce43d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80b1df0 con 0x7f65c80b19d0 2023-07-06T02:24:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65ce43d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8156e80 con 0x7f65c81600b0 2023-07-06T02:24:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65c7fff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f65c8156ab0 0x7f65c8160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59708/0 (socket says 172.21.15.89:59708) 2023-07-06T02:24:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65c7fff700 1 -- 172.21.15.89:0/1147675967 learned_addr learned my addr 172.21.15.89:0/1147675967 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843997818 0 0) 0x7f65c80cb3d0 con 0x7f65c8156ab0 2023-07-06T02:24:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b00035b0 con 0x7f65c8156ab0 2023-07-06T02:24:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65c77fe700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 0x7f65c80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108967511 0 0) 0x7f65b00035b0 con 0x7f65c8156ab0 2023-07-06T02:24:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.791+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 msgr2=0x7f65c80b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 0x7f65c80b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 >> v1:172.21.15.172:6789/0 conn(0x7f65c80b19d0 legacy=0x7f65c80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80ca370 con 0x7f65c8156ab0 2023-07-06T02:24:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f65bc001670 con 0x7f65c8156ab0 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 >> v1:172.21.15.89:6789/0 conn(0x7f65c8156ab0 legacy=0x7f65c8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 0x7f65b0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b0004d90 0x7f65b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8156e80 con 0x7f65b0003fb0 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b000baf0 con 0x7f65b0004d90 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b000bcb0 con 0x7f65c81600b0 2023-07-06T02:24:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c77fe700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c7fff700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 0x7f65b0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65cc9da700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b0004d90 0x7f65b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c77fe700 1 -- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 msgr2=0x7f65b0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c77fe700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c81600b0 0x7f65b0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c77fe700 1 -- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b0004d90 msgr2=0x7f65b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c77fe700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b0004d90 0x7f65b0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c77fe700 1 -- 172.21.15.89:0/1147675967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00035b0 con 0x7f65b0003fb0 2023-07-06T02:24:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c77fe700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65b0004850 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009b30 tx=0x7f65b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ecbd365943fda9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800e6c0 con 0x7f65b0003fb0 2023-07-06T02:24:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b800e820 con 0x7f65b0003fb0 2023-07-06T02:24:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.792+0000 7f65c6ffd700 1 -- 172.21.15.89:0/1147675967 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800eb90 con 0x7f65b0003fb0 2023-07-06T02:24:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.793+0000 7f65ce43d700 1 -- 172.21.15.89:0/1147675967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 msgr2=0x7f65b0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.793+0000 7f65ce43d700 1 --2- 172.21.15.89:0/1147675967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65b0004850 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009b30 tx=0x7f65b0005880 comp rx=0 tx=0).stop 2023-07-06T02:24:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.793+0000 7f65cc9da700 1 -- 172.21.15.89:0/1147675967 reap_dead start 2023-07-06T02:24:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.793+0000 7f65ce43d700 1 -- 172.21.15.89:0/1147675967 shutdown_connections 2023-07-06T02:24:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.793+0000 7f65ce43d700 1 -- 172.21.15.89:0/1147675967 >> 172.21.15.89:0/1147675967 conn(0x7f65c80cc490 msgr2=0x7f65c80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.793+0000 7f65ce43d700 1 -- 172.21.15.89:0/1147675967 shutdown_connections 2023-07-06T02:24:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.793+0000 7f65ce43d700 1 -- 172.21.15.89:0/1147675967 wait complete. 2023-07-06T02:24:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.794+0000 7f65ce43d700 1 Processor -- start 2023-07-06T02:24:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.794+0000 7f65ce43d700 1 -- start start 2023-07-06T02:24:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65ce43d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65c81f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65ce43d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65c80c5db0 0x7f65c80b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65ce43d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c80c6220 0x7f65c815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65ce43d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81f5990 con 0x7f65b0003fb0 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65ce43d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80b3020 con 0x7f65c80c5db0 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65ce43d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815cf50 con 0x7f65c80c6220 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65c81f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65c81f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34818/0 (socket says 172.21.15.89:34818) 2023-07-06T02:24:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 -- 172.21.15.89:0/2547616694 learned_addr learned my addr 172.21.15.89:0/2547616694 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65cc9da700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c80c6220 0x7f65c815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c77fe700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65c80c5db0 0x7f65c80b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 -- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c80c6220 msgr2=0x7f65c815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c80c6220 0x7f65c815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 -- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65c80c5db0 msgr2=0x7f65c80b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65c80c5db0 0x7f65c80b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.795+0000 7f65c7fff700 1 -- 172.21.15.89:0/2547616694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c815cc70 con 0x7f65b0003fb0 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.796+0000 7f65c77fe700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65c80c5db0 0x7f65c80b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.796+0000 7f65c7fff700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65c81f5e20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000fb0 tx=0x7f65bc0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.796+0000 7f65c4ff9700 1 -- 172.21.15.89:0/2547616694 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00f6c0 con 0x7f65b0003fb0 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.796+0000 7f65c4ff9700 1 -- 172.21.15.89:0/2547616694 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc00f840 con 0x7f65b0003fb0 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.796+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c81f5340 con 0x7f65b0003fb0 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.796+0000 7f65c4ff9700 1 -- 172.21.15.89:0/2547616694 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00fbf0 con 0x7f65b0003fb0 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.796+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c80b5620 con 0x7f65b0003fb0 2023-07-06T02:24:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.797+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c81f5c70 con 0x7f65b0003fb0 2023-07-06T02:24:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.797+0000 7f65c4ff9700 1 -- 172.21.15.89:0/2547616694 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65bc00fd50 con 0x7f65b0003fb0 2023-07-06T02:24:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.798+0000 7f65c4ff9700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65b406ace0 0x7f65b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.798+0000 7f65c77fe700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65b406ace0 0x7f65b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.798+0000 7f65c4ff9700 1 -- 172.21.15.89:0/2547616694 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f65bc0895d0 con 0x7f65b0003fb0 2023-07-06T02:24:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.798+0000 7f65c77fe700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65b406ace0 0x7f65b406d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65c815f2f0 tx=0x7f65b8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.802+0000 7f65c4ff9700 1 -- 172.21.15.89:0/2547616694 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65bc054540 con 0x7f65b0003fb0 2023-07-06T02:24:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.987+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 --> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65c80ca610 con 0x7f65b406ace0 2023-07-06T02:24:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.995+0000 7f65c4ff9700 1 -- 172.21.15.89:0/2547616694 <== mgr.4100 v2:172.21.15.89:6832/3591115711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311652 (secure 0 0 0) 0x7f65b800e990 con 0x7f65b406ace0 2023-07-06T02:24:15.001 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65b406ace0 msgr2=0x7f65b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65b406ace0 0x7f65b406d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65c815f2f0 tx=0x7f65b8008030 comp rx=0 tx=0).stop 2023-07-06T02:24:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 msgr2=0x7f65c81f5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65c81f5e20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000fb0 tx=0x7f65bc0075f0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 shutdown_connections 2023-07-06T02:24:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65c80c6220 0x7f65c815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f65b406ace0 0x7f65b406d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65b0003fb0 0x7f65c81f5e20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 --2- 172.21.15.89:0/2547616694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65c80c5db0 0x7f65c80b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 >> 172.21.15.89:0/2547616694 conn(0x7f65c80cc490 msgr2=0x7f65c80ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 shutdown_connections 2023-07-06T02:24:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:14.998+0000 7f65ce43d700 1 -- 172.21.15.89:0/2547616694 wait complete. 2023-07-06T02:24:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-07-06T02:24:15.013 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-07-06T02:24:13.997937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220084,"kb_used_data":4468,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754636,"statfs":{"total":773094113280,"available":772868747264,"internally_reserved":0,"allocated":4575232,"data_stored":2677089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.494951"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555122+0000","last_change":"2023-07-06T02:24:10.547574+0000","last_active":"2023-07-06T02:24:10.555122+0000","last_peered":"2023-07-06T02:24:10.555122+0000","last_clean":"2023-07-06T02:24:10.555122+0000","last_became_active":"2023-07-06T02:24:10.547297+0000","last_became_peered":"2023-07-06T02:24:10.547297+0000","last_unstale":"2023-07-06T02:24:10.555122+0000","last_undegraded":"2023-07-06T02:24:10.555122+0000","last_fullsized":"2023-07-06T02:24:10.555122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:06:57.781915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554360+0000","last_change":"2023-07-06T02:24:10.550764+0000","last_active":"2023-07-06T02:24:10.554360+0000","last_peered":"2023-07-06T02:24:10.554360+0000","last_clean":"2023-07-06T02:24:10.554360+0000","last_became_active":"2023-07-06T02:24:10.547108+0000","last_became_peered":"2023-07-06T02:24:10.547108+0000","last_unstale":"2023-07-06T02:24:10.554360+0000","last_undegraded":"2023-07-06T02:24:10.554360+0000","last_fullsized":"2023-07-06T02:24:10.554360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:46:49.223079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551930+0000","last_change":"2023-07-06T02:24:10.545423+0000","last_active":"2023-07-06T02:24:10.551930+0000","last_peered":"2023-07-06T02:24:10.551930+0000","last_clean":"2023-07-06T02:24:10.551930+0000","last_became_active":"2023-07-06T02:24:10.545106+0000","last_became_peered":"2023-07-06T02:24:10.545106+0000","last_unstale":"2023-07-06T02:24:10.551930+0000","last_undegraded":"2023-07-06T02:24:10.551930+0000","last_fullsized":"2023-07-06T02:24:10.551930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:41.253417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557018+0000","last_change":"2023-07-06T02:24:10.542953+0000","last_active":"2023-07-06T02:24:10.557018+0000","last_peered":"2023-07-06T02:24:10.557018+0000","last_clean":"2023-07-06T02:24:10.557018+0000","last_became_active":"2023-07-06T02:24:10.542695+0000","last_became_peered":"2023-07-06T02:24:10.542695+0000","last_unstale":"2023-07-06T02:24:10.557018+0000","last_undegraded":"2023-07-06T02:24:10.557018+0000","last_fullsized":"2023-07-06T02:24:10.557018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:49:19.263018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554233+0000","last_change":"2023-07-06T02:24:10.546797+0000","last_active":"2023-07-06T02:24:10.554233+0000","last_peered":"2023-07-06T02:24:10.554233+0000","last_clean":"2023-07-06T02:24:10.554233+0000","last_became_active":"2023-07-06T02:24:10.546539+0000","last_became_peered":"2023-07-06T02:24:10.546539+0000","last_unstale":"2023-07-06T02:24:10.554233+0000","last_undegraded":"2023-07-06T02:24:10.554233+0000","last_fullsized":"2023-07-06T02:24:10.554233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:53.708345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.558843+0000","last_change":"2023-07-06T02:24:10.548795+0000","last_active":"2023-07-06T02:24:10.558843+0000","last_peered":"2023-07-06T02:24:10.558843+0000","last_clean":"2023-07-06T02:24:10.558843+0000","last_became_active":"2023-07-06T02:24:10.548511+0000","last_became_peered":"2023-07-06T02:24:10.548511+0000","last_unstale":"2023-07-06T02:24:10.558843+0000","last_undegraded":"2023-07-06T02:24:10.558843+0000","last_fullsized":"2023-07-06T02:24:10.558843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:54.087952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.548247+0000","last_change":"2023-07-06T02:24:10.544377+0000","last_active":"2023-07-06T02:24:10.548247+0000","last_peered":"2023-07-06T02:24:10.548247+0000","last_clean":"2023-07-06T02:24:10.548247+0000","last_became_active":"2023-07-06T02:24:10.544130+0000","last_became_peered":"2023-07-06T02:24:10.544130+0000","last_unstale":"2023-07-06T02:24:10.548247+0000","last_undegraded":"2023-07-06T02:24:10.548247+0000","last_fullsized":"2023-07-06T02:24:10.548247+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:29.993759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557534+0000","last_change":"2023-07-06T02:24:10.547933+0000","last_active":"2023-07-06T02:24:10.557534+0000","last_peered":"2023-07-06T02:24:10.557534+0000","last_clean":"2023-07-06T02:24:10.557534+0000","last_became_active":"2023-07-06T02:24:10.547360+0000","last_became_peered":"2023-07-06T02:24:10.547360+0000","last_unstale":"2023-07-06T02:24:10.557534+0000","last_undegraded":"2023-07-06T02:24:10.557534+0000","last_fullsized":"2023-07-06T02:24:10.557534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:33:16.750479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560451+0000","last_change":"2023-07-06T02:24:10.553189+0000","last_active":"2023-07-06T02:24:10.560451+0000","last_peered":"2023-07-06T02:24:10.560451+0000","last_clean":"2023-07-06T02:24:10.560451+0000","last_became_active":"2023-07-06T02:24:10.552894+0000","last_became_peered":"2023-07-06T02:24:10.552894+0000","last_unstale":"2023-07-06T02:24:10.560451+0000","last_undegraded":"2023-07-06T02:24:10.560451+0000","last_fullsized":"2023-07-06T02:24:10.560451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:02.469016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557210+0000","last_change":"2023-07-06T02:24:10.546568+0000","last_active":"2023-07-06T02:24:10.557210+0000","last_peered":"2023-07-06T02:24:10.557210+0000","last_clean":"2023-07-06T02:24:10.557210+0000","last_became_active":"2023-07-06T02:24:10.546114+0000","last_became_peered":"2023-07-06T02:24:10.546114+0000","last_unstale":"2023-07-06T02:24:10.557210+0000","last_undegraded":"2023-07-06T02:24:10.557210+0000","last_fullsized":"2023-07-06T02:24:10.557210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:08.271075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560620+0000","last_change":"2023-07-06T02:24:10.553581+0000","last_active":"2023-07-06T02:24:10.560620+0000","last_peered":"2023-07-06T02:24:10.560620+0000","last_clean":"2023-07-06T02:24:10.560620+0000","last_became_active":"2023-07-06T02:24:10.553310+0000","last_became_peered":"2023-07-06T02:24:10.553310+0000","last_unstale":"2023-07-06T02:24:10.560620+0000","last_undegraded":"2023-07-06T02:24:10.560620+0000","last_fullsized":"2023-07-06T02:24:10.560620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:58:24.057393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541810+0000","last_change":"2023-07-06T02:24:10.537491+0000","last_active":"2023-07-06T02:24:10.541810+0000","last_peered":"2023-07-06T02:24:10.541810+0000","last_clean":"2023-07-06T02:24:10.541810+0000","last_became_active":"2023-07-06T02:24:10.536814+0000","last_became_peered":"2023-07-06T02:24:10.536814+0000","last_unstale":"2023-07-06T02:24:10.541810+0000","last_undegraded":"2023-07-06T02:24:10.541810+0000","last_fullsized":"2023-07-06T02:24:10.541810+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:11.003933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560432+0000","last_change":"2023-07-06T02:24:10.557878+0000","last_active":"2023-07-06T02:24:10.560432+0000","last_peered":"2023-07-06T02:24:10.560432+0000","last_clean":"2023-07-06T02:24:10.560432+0000","last_became_active":"2023-07-06T02:24:10.557600+0000","last_became_peered":"2023-07-06T02:24:10.557600+0000","last_unstale":"2023-07-06T02:24:10.560432+0000","last_undegraded":"2023-07-06T02:24:10.560432+0000","last_fullsized":"2023-07-06T02:24:10.560432+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:04:51.093024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559572+0000","last_change":"2023-07-06T02:24:10.550524+0000","last_active":"2023-07-06T02:24:10.559572+0000","last_peered":"2023-07-06T02:24:10.559572+0000","last_clean":"2023-07-06T02:24:10.559572+0000","last_became_active":"2023-07-06T02:24:10.550255+0000","last_became_peered":"2023-07-06T02:24:10.550255+0000","last_unstale":"2023-07-06T02:24:10.559572+0000","last_undegraded":"2023-07-06T02:24:10.559572+0000","last_fullsized":"2023-07-06T02:24:10.559572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:39:34.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.558319+0000","last_change":"2023-07-06T02:24:10.550141+0000","last_active":"2023-07-06T02:24:10.558319+0000","last_peered":"2023-07-06T02:24:10.558319+0000","last_clean":"2023-07-06T02:24:10.558319+0000","last_became_active":"2023-07-06T02:24:10.549877+0000","last_became_peered":"2023-07-06T02:24:10.549877+0000","last_unstale":"2023-07-06T02:24:10.558319+0000","last_undegraded":"2023-07-06T02:24:10.558319+0000","last_fullsized":"2023-07-06T02:24:10.558319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:03.761474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555823+0000","last_change":"2023-07-06T02:24:10.553912+0000","last_active":"2023-07-06T02:24:10.555823+0000","last_peered":"2023-07-06T02:24:10.555823+0000","last_clean":"2023-07-06T02:24:10.555823+0000","last_became_active":"2023-07-06T02:24:10.553558+0000","last_became_peered":"2023-07-06T02:24:10.553558+0000","last_unstale":"2023-07-06T02:24:10.555823+0000","last_undegraded":"2023-07-06T02:24:10.555823+0000","last_fullsized":"2023-07-06T02:24:10.555823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:52:08.212360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551959+0000","last_change":"2023-07-06T02:24:10.546095+0000","last_active":"2023-07-06T02:24:10.551959+0000","last_peered":"2023-07-06T02:24:10.551959+0000","last_clean":"2023-07-06T02:24:10.551959+0000","last_became_active":"2023-07-06T02:24:10.544582+0000","last_became_peered":"2023-07-06T02:24:10.544582+0000","last_unstale":"2023-07-06T02:24:10.551959+0000","last_undegraded":"2023-07-06T02:24:10.551959+0000","last_fullsized":"2023-07-06T02:24:10.551959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:55:37.425912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554657+0000","last_change":"2023-07-06T02:24:10.547888+0000","last_active":"2023-07-06T02:24:10.554657+0000","last_peered":"2023-07-06T02:24:10.554657+0000","last_clean":"2023-07-06T02:24:10.554657+0000","last_became_active":"2023-07-06T02:24:10.545860+0000","last_became_peered":"2023-07-06T02:24:10.545860+0000","last_unstale":"2023-07-06T02:24:10.554657+0000","last_undegraded":"2023-07-06T02:24:10.554657+0000","last_fullsized":"2023-07-06T02:24:10.554657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:24.271903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.566255+0000","last_change":"2023-07-06T02:24:10.557841+0000","last_active":"2023-07-06T02:24:10.566255+0000","last_peered":"2023-07-06T02:24:10.566255+0000","last_clean":"2023-07-06T02:24:10.566255+0000","last_became_active":"2023-07-06T02:24:10.557614+0000","last_became_peered":"2023-07-06T02:24:10.557614+0000","last_unstale":"2023-07-06T02:24:10.566255+0000","last_undegraded":"2023-07-06T02:24:10.566255+0000","last_fullsized":"2023-07-06T02:24:10.566255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:23:11.474183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551568+0000","last_change":"2023-07-06T02:24:10.548573+0000","last_active":"2023-07-06T02:24:10.551568+0000","last_peered":"2023-07-06T02:24:10.551568+0000","last_clean":"2023-07-06T02:24:10.551568+0000","last_became_active":"2023-07-06T02:24:10.547705+0000","last_became_peered":"2023-07-06T02:24:10.547705+0000","last_unstale":"2023-07-06T02:24:10.551568+0000","last_undegraded":"2023-07-06T02:24:10.551568+0000","last_fullsized":"2023-07-06T02:24:10.551568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:54:52.631910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561772+0000","last_change":"2023-07-06T02:24:10.555704+0000","last_active":"2023-07-06T02:24:10.561772+0000","last_peered":"2023-07-06T02:24:10.561772+0000","last_clean":"2023-07-06T02:24:10.561772+0000","last_became_active":"2023-07-06T02:24:10.555555+0000","last_became_peered":"2023-07-06T02:24:10.555555+0000","last_unstale":"2023-07-06T02:24:10.561772+0000","last_undegraded":"2023-07-06T02:24:10.561772+0000","last_fullsized":"2023-07-06T02:24:10.561772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:21:40.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.556908+0000","last_change":"2023-07-06T02:24:10.544803+0000","last_active":"2023-07-06T02:24:10.556908+0000","last_peered":"2023-07-06T02:24:10.556908+0000","last_clean":"2023-07-06T02:24:10.556908+0000","last_became_active":"2023-07-06T02:24:10.544478+0000","last_became_peered":"2023-07-06T02:24:10.544478+0000","last_unstale":"2023-07-06T02:24:10.556908+0000","last_undegraded":"2023-07-06T02:24:10.556908+0000","last_fullsized":"2023-07-06T02:24:10.556908+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:43.358611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555015+0000","last_change":"2023-07-06T02:24:10.549075+0000","last_active":"2023-07-06T02:24:10.555015+0000","last_peered":"2023-07-06T02:24:10.555015+0000","last_clean":"2023-07-06T02:24:10.555015+0000","last_became_active":"2023-07-06T02:24:10.548634+0000","last_became_peered":"2023-07-06T02:24:10.548634+0000","last_unstale":"2023-07-06T02:24:10.555015+0000","last_undegraded":"2023-07-06T02:24:10.555015+0000","last_fullsized":"2023-07-06T02:24:10.555015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:58.637874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554901+0000","last_change":"2023-07-06T02:24:10.545014+0000","last_active":"2023-07-06T02:24:10.554901+0000","last_peered":"2023-07-06T02:24:10.554901+0000","last_clean":"2023-07-06T02:24:10.554901+0000","last_became_active":"2023-07-06T02:24:10.544701+0000","last_became_peered":"2023-07-06T02:24:10.544701+0000","last_unstale":"2023-07-06T02:24:10.554901+0000","last_undegraded":"2023-07-06T02:24:10.554901+0000","last_fullsized":"2023-07-06T02:24:10.554901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:40:00.950400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554849+0000","last_change":"2023-07-06T02:24:10.546635+0000","last_active":"2023-07-06T02:24:10.554849+0000","last_peered":"2023-07-06T02:24:10.554849+0000","last_clean":"2023-07-06T02:24:10.554849+0000","last_became_active":"2023-07-06T02:24:10.546370+0000","last_became_peered":"2023-07-06T02:24:10.546370+0000","last_unstale":"2023-07-06T02:24:10.554849+0000","last_undegraded":"2023-07-06T02:24:10.554849+0000","last_fullsized":"2023-07-06T02:24:10.554849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:50:45.017398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554732+0000","last_change":"2023-07-06T02:24:10.550324+0000","last_active":"2023-07-06T02:24:10.554732+0000","last_peered":"2023-07-06T02:24:10.554732+0000","last_clean":"2023-07-06T02:24:10.554732+0000","last_became_active":"2023-07-06T02:24:10.550042+0000","last_became_peered":"2023-07-06T02:24:10.550042+0000","last_unstale":"2023-07-06T02:24:10.554732+0000","last_undegraded":"2023-07-06T02:24:10.554732+0000","last_fullsized":"2023-07-06T02:24:10.554732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:29.095011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551903+0000","last_change":"2023-07-06T02:24:10.543676+0000","last_active":"2023-07-06T02:24:10.551903+0000","last_peered":"2023-07-06T02:24:10.551903+0000","last_clean":"2023-07-06T02:24:10.551903+0000","last_became_active":"2023-07-06T02:24:10.543400+0000","last_became_peered":"2023-07-06T02:24:10.543400+0000","last_unstale":"2023-07-06T02:24:10.551903+0000","last_undegraded":"2023-07-06T02:24:10.551903+0000","last_fullsized":"2023-07-06T02:24:10.551903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:26.079979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.542343+0000","last_change":"2023-07-06T02:24:10.537667+0000","last_active":"2023-07-06T02:24:10.542343+0000","last_peered":"2023-07-06T02:24:10.542343+0000","last_clean":"2023-07-06T02:24:10.542343+0000","last_became_active":"2023-07-06T02:24:10.537203+0000","last_became_peered":"2023-07-06T02:24:10.537203+0000","last_unstale":"2023-07-06T02:24:10.542343+0000","last_undegraded":"2023-07-06T02:24:10.542343+0000","last_fullsized":"2023-07-06T02:24:10.542343+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:35:42.843589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.553030+0000","last_change":"2023-07-06T02:24:10.545904+0000","last_active":"2023-07-06T02:24:10.553030+0000","last_peered":"2023-07-06T02:24:10.553030+0000","last_clean":"2023-07-06T02:24:10.553030+0000","last_became_active":"2023-07-06T02:24:10.545652+0000","last_became_peered":"2023-07-06T02:24:10.545652+0000","last_unstale":"2023-07-06T02:24:10.553030+0000","last_undegraded":"2023-07-06T02:24:10.553030+0000","last_fullsized":"2023-07-06T02:24:10.553030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:03:43.429442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551991+0000","last_change":"2023-07-06T02:24:10.543645+0000","last_active":"2023-07-06T02:24:10.551991+0000","last_peered":"2023-07-06T02:24:10.551991+0000","last_clean":"2023-07-06T02:24:10.551991+0000","last_became_active":"2023-07-06T02:24:10.543355+0000","last_became_peered":"2023-07-06T02:24:10.543355+0000","last_unstale":"2023-07-06T02:24:10.551991+0000","last_undegraded":"2023-07-06T02:24:10.551991+0000","last_fullsized":"2023-07-06T02:24:10.551991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:35.243003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561833+0000","last_change":"2023-07-06T02:24:10.558542+0000","last_active":"2023-07-06T02:24:10.561833+0000","last_peered":"2023-07-06T02:24:10.561833+0000","last_clean":"2023-07-06T02:24:10.561833+0000","last_became_active":"2023-07-06T02:24:10.558247+0000","last_became_peered":"2023-07-06T02:24:10.558247+0000","last_unstale":"2023-07-06T02:24:10.561833+0000","last_undegraded":"2023-07-06T02:24:10.561833+0000","last_fullsized":"2023-07-06T02:24:10.561833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:14.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.549350+0000","last_change":"2023-07-06T02:24:10.545571+0000","last_active":"2023-07-06T02:24:10.549350+0000","last_peered":"2023-07-06T02:24:10.549350+0000","last_clean":"2023-07-06T02:24:10.549350+0000","last_became_active":"2023-07-06T02:24:10.545274+0000","last_became_peered":"2023-07-06T02:24:10.545274+0000","last_unstale":"2023-07-06T02:24:10.549350+0000","last_undegraded":"2023-07-06T02:24:10.549350+0000","last_fullsized":"2023-07-06T02:24:10.549350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:01.368463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554097+0000","last_change":"2023-07-06T02:24:10.551064+0000","last_active":"2023-07-06T02:24:10.554097+0000","last_peered":"2023-07-06T02:24:10.554097+0000","last_clean":"2023-07-06T02:24:10.554097+0000","last_became_active":"2023-07-06T02:24:10.550787+0000","last_became_peered":"2023-07-06T02:24:10.550787+0000","last_unstale":"2023-07-06T02:24:10.554097+0000","last_undegraded":"2023-07-06T02:24:10.554097+0000","last_fullsized":"2023-07-06T02:24:10.554097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:35:47.914273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559239+0000","last_change":"2023-07-06T02:24:09.524883+0000","last_active":"2023-07-06T02:24:10.559239+0000","last_peered":"2023-07-06T02:24:10.559239+0000","last_clean":"2023-07-06T02:24:10.559239+0000","last_became_active":"2023-07-06T02:24:09.524444+0000","last_became_peered":"2023-07-06T02:24:09.524444+0000","last_unstale":"2023-07-06T02:24:10.559239+0000","last_undegraded":"2023-07-06T02:24:10.559239+0000","last_fullsized":"2023-07-06T02:24:10.559239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:15:29.718817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554935+0000","last_change":"2023-07-06T02:24:10.547891+0000","last_active":"2023-07-06T02:24:10.554935+0000","last_peered":"2023-07-06T02:24:10.554935+0000","last_clean":"2023-07-06T02:24:10.554935+0000","last_became_active":"2023-07-06T02:24:10.547595+0000","last_became_peered":"2023-07-06T02:24:10.547595+0000","last_unstale":"2023-07-06T02:24:10.554935+0000","last_undegraded":"2023-07-06T02:24:10.554935+0000","last_fullsized":"2023-07-06T02:24:10.554935+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:02:30.508728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.548005+0000","last_change":"2023-07-06T02:24:09.526186+0000","last_active":"2023-07-06T02:24:10.548005+0000","last_peered":"2023-07-06T02:24:10.548005+0000","last_clean":"2023-07-06T02:24:10.548005+0000","last_became_active":"2023-07-06T02:24:09.525901+0000","last_became_peered":"2023-07-06T02:24:09.525901+0000","last_unstale":"2023-07-06T02:24:10.548005+0000","last_undegraded":"2023-07-06T02:24:10.548005+0000","last_fullsized":"2023-07-06T02:24:10.548005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:17.022973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554982+0000","last_change":"2023-07-06T02:24:10.552868+0000","last_active":"2023-07-06T02:24:10.554982+0000","last_peered":"2023-07-06T02:24:10.554982+0000","last_clean":"2023-07-06T02:24:10.554982+0000","last_became_active":"2023-07-06T02:24:10.552616+0000","last_became_peered":"2023-07-06T02:24:10.552616+0000","last_unstale":"2023-07-06T02:24:10.554982+0000","last_undegraded":"2023-07-06T02:24:10.554982+0000","last_fullsized":"2023-07-06T02:24:10.554982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:44.486164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551941+0000","last_change":"2023-07-06T02:24:09.525143+0000","last_active":"2023-07-06T02:24:10.551941+0000","last_peered":"2023-07-06T02:24:10.551941+0000","last_clean":"2023-07-06T02:24:10.551941+0000","last_became_active":"2023-07-06T02:24:09.523406+0000","last_became_peered":"2023-07-06T02:24:09.523406+0000","last_unstale":"2023-07-06T02:24:10.551941+0000","last_undegraded":"2023-07-06T02:24:10.551941+0000","last_fullsized":"2023-07-06T02:24:10.551941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:15.941505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554137+0000","last_change":"2023-07-06T02:24:10.551787+0000","last_active":"2023-07-06T02:24:10.554137+0000","last_peered":"2023-07-06T02:24:10.554137+0000","last_clean":"2023-07-06T02:24:10.554137+0000","last_became_active":"2023-07-06T02:24:10.551418+0000","last_became_peered":"2023-07-06T02:24:10.551418+0000","last_unstale":"2023-07-06T02:24:10.554137+0000","last_undegraded":"2023-07-06T02:24:10.554137+0000","last_fullsized":"2023-07-06T02:24:10.554137+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:52:58.133143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547159+0000","last_change":"2023-07-06T02:24:09.528002+0000","last_active":"2023-07-06T02:24:10.547159+0000","last_peered":"2023-07-06T02:24:10.547159+0000","last_clean":"2023-07-06T02:24:10.547159+0000","last_became_active":"2023-07-06T02:24:09.527699+0000","last_became_peered":"2023-07-06T02:24:09.527699+0000","last_unstale":"2023-07-06T02:24:10.547159+0000","last_undegraded":"2023-07-06T02:24:10.547159+0000","last_fullsized":"2023-07-06T02:24:10.547159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:34:02.328706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.552010+0000","last_change":"2023-07-06T02:24:10.545453+0000","last_active":"2023-07-06T02:24:10.552010+0000","last_peered":"2023-07-06T02:24:10.552010+0000","last_clean":"2023-07-06T02:24:10.552010+0000","last_became_active":"2023-07-06T02:24:10.545090+0000","last_became_peered":"2023-07-06T02:24:10.545090+0000","last_unstale":"2023-07-06T02:24:10.552010+0000","last_undegraded":"2023-07-06T02:24:10.552010+0000","last_fullsized":"2023-07-06T02:24:10.552010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:29:51.253224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560675+0000","last_change":"2023-07-06T02:24:09.522080+0000","last_active":"2023-07-06T02:24:10.560675+0000","last_peered":"2023-07-06T02:24:10.560675+0000","last_clean":"2023-07-06T02:24:10.560675+0000","last_became_active":"2023-07-06T02:24:09.521312+0000","last_became_peered":"2023-07-06T02:24:09.521312+0000","last_unstale":"2023-07-06T02:24:10.560675+0000","last_undegraded":"2023-07-06T02:24:10.560675+0000","last_fullsized":"2023-07-06T02:24:10.560675+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:25.475842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559208+0000","last_change":"2023-07-06T02:24:10.548867+0000","last_active":"2023-07-06T02:24:10.559208+0000","last_peered":"2023-07-06T02:24:10.559208+0000","last_clean":"2023-07-06T02:24:10.559208+0000","last_became_active":"2023-07-06T02:24:10.548427+0000","last_became_peered":"2023-07-06T02:24:10.548427+0000","last_unstale":"2023-07-06T02:24:10.559208+0000","last_undegraded":"2023-07-06T02:24:10.559208+0000","last_fullsized":"2023-07-06T02:24:10.559208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:18:22.869326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547350+0000","last_change":"2023-07-06T02:24:09.523878+0000","last_active":"2023-07-06T02:24:10.547350+0000","last_peered":"2023-07-06T02:24:10.547350+0000","last_clean":"2023-07-06T02:24:10.547350+0000","last_became_active":"2023-07-06T02:24:09.523621+0000","last_became_peered":"2023-07-06T02:24:09.523621+0000","last_unstale":"2023-07-06T02:24:10.547350+0000","last_undegraded":"2023-07-06T02:24:10.547350+0000","last_fullsized":"2023-07-06T02:24:10.547350+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:17.177698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557553+0000","last_change":"2023-07-06T02:24:10.548079+0000","last_active":"2023-07-06T02:24:10.557553+0000","last_peered":"2023-07-06T02:24:10.557553+0000","last_clean":"2023-07-06T02:24:10.557553+0000","last_became_active":"2023-07-06T02:24:10.547630+0000","last_became_peered":"2023-07-06T02:24:10.547630+0000","last_unstale":"2023-07-06T02:24:10.557553+0000","last_undegraded":"2023-07-06T02:24:10.557553+0000","last_fullsized":"2023-07-06T02:24:10.557553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:53:13.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.526573+0000","last_change":"2023-07-06T02:24:09.523476+0000","last_active":"2023-07-06T02:24:10.526573+0000","last_peered":"2023-07-06T02:24:10.526573+0000","last_clean":"2023-07-06T02:24:10.526573+0000","last_became_active":"2023-07-06T02:24:09.523172+0000","last_became_peered":"2023-07-06T02:24:09.523172+0000","last_unstale":"2023-07-06T02:24:10.526573+0000","last_undegraded":"2023-07-06T02:24:10.526573+0000","last_fullsized":"2023-07-06T02:24:10.526573+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:38:10.219049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555034+0000","last_change":"2023-07-06T02:24:10.549234+0000","last_active":"2023-07-06T02:24:10.555034+0000","last_peered":"2023-07-06T02:24:10.555034+0000","last_clean":"2023-07-06T02:24:10.555034+0000","last_became_active":"2023-07-06T02:24:10.548920+0000","last_became_peered":"2023-07-06T02:24:10.548920+0000","last_unstale":"2023-07-06T02:24:10.555034+0000","last_undegraded":"2023-07-06T02:24:10.555034+0000","last_fullsized":"2023-07-06T02:24:10.555034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:17:14.239545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541311+0000","last_change":"2023-07-06T02:24:09.528575+0000","last_active":"2023-07-06T02:24:10.541311+0000","last_peered":"2023-07-06T02:24:10.541311+0000","last_clean":"2023-07-06T02:24:10.541311+0000","last_became_active":"2023-07-06T02:24:09.528028+0000","last_became_peered":"2023-07-06T02:24:09.528028+0000","last_unstale":"2023-07-06T02:24:10.541311+0000","last_undegraded":"2023-07-06T02:24:10.541311+0000","last_fullsized":"2023-07-06T02:24:10.541311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:38:05.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557641+0000","last_change":"2023-07-06T02:24:10.553773+0000","last_active":"2023-07-06T02:24:10.557641+0000","last_peered":"2023-07-06T02:24:10.557641+0000","last_clean":"2023-07-06T02:24:10.557641+0000","last_became_active":"2023-07-06T02:24:10.553240+0000","last_became_peered":"2023-07-06T02:24:10.553240+0000","last_unstale":"2023-07-06T02:24:10.557641+0000","last_undegraded":"2023-07-06T02:24:10.557641+0000","last_fullsized":"2023-07-06T02:24:10.557641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:21:47.220791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530847+0000","last_change":"2023-07-06T02:24:09.522755+0000","last_active":"2023-07-06T02:24:10.530847+0000","last_peered":"2023-07-06T02:24:10.530847+0000","last_clean":"2023-07-06T02:24:10.530847+0000","last_became_active":"2023-07-06T02:24:09.522270+0000","last_became_peered":"2023-07-06T02:24:09.522270+0000","last_unstale":"2023-07-06T02:24:10.530847+0000","last_undegraded":"2023-07-06T02:24:10.530847+0000","last_fullsized":"2023-07-06T02:24:10.530847+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:23:13.688739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.543043+0000","last_change":"2023-07-06T02:24:10.539321+0000","last_active":"2023-07-06T02:24:10.543043+0000","last_peered":"2023-07-06T02:24:10.543043+0000","last_clean":"2023-07-06T02:24:10.543043+0000","last_became_active":"2023-07-06T02:24:10.539066+0000","last_became_peered":"2023-07-06T02:24:10.539066+0000","last_unstale":"2023-07-06T02:24:10.543043+0000","last_undegraded":"2023-07-06T02:24:10.543043+0000","last_fullsized":"2023-07-06T02:24:10.543043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:21:40.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.543621+0000","last_change":"2023-07-06T02:24:09.530485+0000","last_active":"2023-07-06T02:24:10.543621+0000","last_peered":"2023-07-06T02:24:10.543621+0000","last_clean":"2023-07-06T02:24:10.543621+0000","last_became_active":"2023-07-06T02:24:09.530184+0000","last_became_peered":"2023-07-06T02:24:09.530184+0000","last_unstale":"2023-07-06T02:24:10.543621+0000","last_undegraded":"2023-07-06T02:24:10.543621+0000","last_fullsized":"2023-07-06T02:24:10.543621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:31:06.867675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561970+0000","last_change":"2023-07-06T02:24:10.559022+0000","last_active":"2023-07-06T02:24:10.561970+0000","last_peered":"2023-07-06T02:24:10.561970+0000","last_clean":"2023-07-06T02:24:10.561970+0000","last_became_active":"2023-07-06T02:24:10.558784+0000","last_became_peered":"2023-07-06T02:24:10.558784+0000","last_unstale":"2023-07-06T02:24:10.561970+0000","last_undegraded":"2023-07-06T02:24:10.561970+0000","last_fullsized":"2023-07-06T02:24:10.561970+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:37:08.277626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547366+0000","last_change":"2023-07-06T02:24:09.531056+0000","last_active":"2023-07-06T02:24:10.547366+0000","last_peered":"2023-07-06T02:24:10.547366+0000","last_clean":"2023-07-06T02:24:10.547366+0000","last_became_active":"2023-07-06T02:24:09.530682+0000","last_became_peered":"2023-07-06T02:24:09.530682+0000","last_unstale":"2023-07-06T02:24:10.547366+0000","last_undegraded":"2023-07-06T02:24:10.547366+0000","last_fullsized":"2023-07-06T02:24:10.547366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:37:07.271615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555398+0000","last_change":"2023-07-06T02:24:10.552688+0000","last_active":"2023-07-06T02:24:10.555398+0000","last_peered":"2023-07-06T02:24:10.555398+0000","last_clean":"2023-07-06T02:24:10.555398+0000","last_became_active":"2023-07-06T02:24:10.552418+0000","last_became_peered":"2023-07-06T02:24:10.552418+0000","last_unstale":"2023-07-06T02:24:10.555398+0000","last_undegraded":"2023-07-06T02:24:10.555398+0000","last_fullsized":"2023-07-06T02:24:10.555398+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:34.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.566834+0000","last_change":"2023-07-06T02:24:09.528086+0000","last_active":"2023-07-06T02:24:10.566834+0000","last_peered":"2023-07-06T02:24:10.566834+0000","last_clean":"2023-07-06T02:24:10.566834+0000","last_became_active":"2023-07-06T02:24:09.527768+0000","last_became_peered":"2023-07-06T02:24:09.527768+0000","last_unstale":"2023-07-06T02:24:10.566834+0000","last_undegraded":"2023-07-06T02:24:10.566834+0000","last_fullsized":"2023-07-06T02:24:10.566834+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:30:36.725924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554879+0000","last_change":"2023-07-06T02:24:10.546835+0000","last_active":"2023-07-06T02:24:10.554879+0000","last_peered":"2023-07-06T02:24:10.554879+0000","last_clean":"2023-07-06T02:24:10.554879+0000","last_became_active":"2023-07-06T02:24:10.546533+0000","last_became_peered":"2023-07-06T02:24:10.546533+0000","last_unstale":"2023-07-06T02:24:10.554879+0000","last_undegraded":"2023-07-06T02:24:10.554879+0000","last_fullsized":"2023-07-06T02:24:10.554879+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:57:37.805536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.563265+0000","last_change":"2023-07-06T02:24:09.529989+0000","last_active":"2023-07-06T02:24:10.563265+0000","last_peered":"2023-07-06T02:24:10.563265+0000","last_clean":"2023-07-06T02:24:10.563265+0000","last_became_active":"2023-07-06T02:24:09.529734+0000","last_became_peered":"2023-07-06T02:24:09.529734+0000","last_unstale":"2023-07-06T02:24:10.563265+0000","last_undegraded":"2023-07-06T02:24:10.563265+0000","last_fullsized":"2023-07-06T02:24:10.563265+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:48.336132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559189+0000","last_change":"2023-07-06T02:24:10.546105+0000","last_active":"2023-07-06T02:24:10.559189+0000","last_peered":"2023-07-06T02:24:10.559189+0000","last_clean":"2023-07-06T02:24:10.559189+0000","last_became_active":"2023-07-06T02:24:10.545831+0000","last_became_peered":"2023-07-06T02:24:10.545831+0000","last_unstale":"2023-07-06T02:24:10.559189+0000","last_undegraded":"2023-07-06T02:24:10.559189+0000","last_fullsized":"2023-07-06T02:24:10.559189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:53.030078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.529725+0000","last_change":"2023-07-06T02:24:09.527980+0000","last_active":"2023-07-06T02:24:10.529725+0000","last_peered":"2023-07-06T02:24:10.529725+0000","last_clean":"2023-07-06T02:24:10.529725+0000","last_became_active":"2023-07-06T02:24:09.527641+0000","last_became_peered":"2023-07-06T02:24:09.527641+0000","last_unstale":"2023-07-06T02:24:10.529725+0000","last_undegraded":"2023-07-06T02:24:10.529725+0000","last_fullsized":"2023-07-06T02:24:10.529725+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:31:06.867675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561755+0000","last_change":"2023-07-06T02:24:10.555431+0000","last_active":"2023-07-06T02:24:10.561755+0000","last_peered":"2023-07-06T02:24:10.561755+0000","last_clean":"2023-07-06T02:24:10.561755+0000","last_became_active":"2023-07-06T02:24:10.555108+0000","last_became_peered":"2023-07-06T02:24:10.555108+0000","last_unstale":"2023-07-06T02:24:10.561755+0000","last_undegraded":"2023-07-06T02:24:10.561755+0000","last_fullsized":"2023-07-06T02:24:10.561755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:29.778784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.539655+0000","last_change":"2023-07-06T02:24:09.531563+0000","last_active":"2023-07-06T02:24:10.539655+0000","last_peered":"2023-07-06T02:24:10.539655+0000","last_clean":"2023-07-06T02:24:10.539655+0000","last_became_active":"2023-07-06T02:24:09.531278+0000","last_became_peered":"2023-07-06T02:24:09.531278+0000","last_unstale":"2023-07-06T02:24:10.539655+0000","last_undegraded":"2023-07-06T02:24:10.539655+0000","last_fullsized":"2023-07-06T02:24:10.539655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:06:10.921079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.548236+0000","last_change":"2023-07-06T02:24:10.540350+0000","last_active":"2023-07-06T02:24:10.548236+0000","last_peered":"2023-07-06T02:24:10.548236+0000","last_clean":"2023-07-06T02:24:10.548236+0000","last_became_active":"2023-07-06T02:24:10.540079+0000","last_became_peered":"2023-07-06T02:24:10.540079+0000","last_unstale":"2023-07-06T02:24:10.548236+0000","last_undegraded":"2023-07-06T02:24:10.548236+0000","last_fullsized":"2023-07-06T02:24:10.548236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:14.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559856+0000","last_change":"2023-07-06T02:24:09.528422+0000","last_active":"2023-07-06T02:24:10.559856+0000","last_peered":"2023-07-06T02:24:10.559856+0000","last_clean":"2023-07-06T02:24:10.559856+0000","last_became_active":"2023-07-06T02:24:09.527703+0000","last_became_peered":"2023-07-06T02:24:09.527703+0000","last_unstale":"2023-07-06T02:24:10.559856+0000","last_undegraded":"2023-07-06T02:24:10.559856+0000","last_fullsized":"2023-07-06T02:24:10.559856+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:57:44.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.546950+0000","last_change":"2023-07-06T02:24:10.536109+0000","last_active":"2023-07-06T02:24:10.546950+0000","last_peered":"2023-07-06T02:24:10.546950+0000","last_clean":"2023-07-06T02:24:10.546950+0000","last_became_active":"2023-07-06T02:24:10.535837+0000","last_became_peered":"2023-07-06T02:24:10.535837+0000","last_unstale":"2023-07-06T02:24:10.546950+0000","last_undegraded":"2023-07-06T02:24:10.546950+0000","last_fullsized":"2023-07-06T02:24:10.546950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:47:39.164709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559910+0000","last_change":"2023-07-06T02:24:09.524241+0000","last_active":"2023-07-06T02:24:10.559910+0000","last_peered":"2023-07-06T02:24:10.559910+0000","last_clean":"2023-07-06T02:24:10.559910+0000","last_became_active":"2023-07-06T02:24:09.524006+0000","last_became_peered":"2023-07-06T02:24:09.524006+0000","last_unstale":"2023-07-06T02:24:10.559910+0000","last_undegraded":"2023-07-06T02:24:10.559910+0000","last_fullsized":"2023-07-06T02:24:10.559910+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:16.101315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559764+0000","last_change":"2023-07-06T02:24:10.552211+0000","last_active":"2023-07-06T02:24:10.559764+0000","last_peered":"2023-07-06T02:24:10.559764+0000","last_clean":"2023-07-06T02:24:10.559764+0000","last_became_active":"2023-07-06T02:24:10.551946+0000","last_became_peered":"2023-07-06T02:24:10.551946+0000","last_unstale":"2023-07-06T02:24:10.559764+0000","last_undegraded":"2023-07-06T02:24:10.559764+0000","last_fullsized":"2023-07-06T02:24:10.559764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:51:50.803477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.550023+0000","last_change":"2023-07-06T02:24:09.522250+0000","last_active":"2023-07-06T02:24:10.550023+0000","last_peered":"2023-07-06T02:24:10.550023+0000","last_clean":"2023-07-06T02:24:10.550023+0000","last_became_active":"2023-07-06T02:24:09.521482+0000","last_became_peered":"2023-07-06T02:24:09.521482+0000","last_unstale":"2023-07-06T02:24:10.550023+0000","last_undegraded":"2023-07-06T02:24:10.550023+0000","last_fullsized":"2023-07-06T02:24:10.550023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:06.553635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555845+0000","last_change":"2023-07-06T02:24:10.554357+0000","last_active":"2023-07-06T02:24:10.555845+0000","last_peered":"2023-07-06T02:24:10.555845+0000","last_clean":"2023-07-06T02:24:10.555845+0000","last_became_active":"2023-07-06T02:24:10.554046+0000","last_became_peered":"2023-07-06T02:24:10.554046+0000","last_unstale":"2023-07-06T02:24:10.555845+0000","last_undegraded":"2023-07-06T02:24:10.555845+0000","last_fullsized":"2023-07-06T02:24:10.555845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:29:26.733224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.551785+0000","last_change":"2023-07-06T02:24:09.523544+0000","last_active":"2023-07-06T02:24:10.551785+0000","last_peered":"2023-07-06T02:24:10.551785+0000","last_clean":"2023-07-06T02:24:10.551785+0000","last_became_active":"2023-07-06T02:24:09.523242+0000","last_became_peered":"2023-07-06T02:24:09.523242+0000","last_unstale":"2023-07-06T02:24:10.551785+0000","last_undegraded":"2023-07-06T02:24:10.551785+0000","last_fullsized":"2023-07-06T02:24:10.551785+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:18:23.789530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555697+0000","last_change":"2023-07-06T02:24:10.552869+0000","last_active":"2023-07-06T02:24:10.555697+0000","last_peered":"2023-07-06T02:24:10.555697+0000","last_clean":"2023-07-06T02:24:10.555697+0000","last_became_active":"2023-07-06T02:24:10.552542+0000","last_became_peered":"2023-07-06T02:24:10.552542+0000","last_unstale":"2023-07-06T02:24:10.555697+0000","last_undegraded":"2023-07-06T02:24:10.555697+0000","last_fullsized":"2023-07-06T02:24:10.555697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:45.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.537819+0000","last_change":"2023-07-06T02:24:09.528988+0000","last_active":"2023-07-06T02:24:10.537819+0000","last_peered":"2023-07-06T02:24:10.537819+0000","last_clean":"2023-07-06T02:24:10.537819+0000","last_became_active":"2023-07-06T02:24:09.528698+0000","last_became_peered":"2023-07-06T02:24:09.528698+0000","last_unstale":"2023-07-06T02:24:10.537819+0000","last_undegraded":"2023-07-06T02:24:10.537819+0000","last_fullsized":"2023-07-06T02:24:10.537819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:38.533140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561846+0000","last_change":"2023-07-06T02:24:10.558634+0000","last_active":"2023-07-06T02:24:10.561846+0000","last_peered":"2023-07-06T02:24:10.561846+0000","last_clean":"2023-07-06T02:24:10.561846+0000","last_became_active":"2023-07-06T02:24:10.558377+0000","last_became_peered":"2023-07-06T02:24:10.558377+0000","last_unstale":"2023-07-06T02:24:10.561846+0000","last_undegraded":"2023-07-06T02:24:10.561846+0000","last_fullsized":"2023-07-06T02:24:10.561846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:34.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.561486+0000","last_change":"2023-07-06T02:24:09.522239+0000","last_active":"2023-07-06T02:24:10.561486+0000","last_peered":"2023-07-06T02:24:10.561486+0000","last_clean":"2023-07-06T02:24:10.561486+0000","last_became_active":"2023-07-06T02:24:09.521822+0000","last_became_peered":"2023-07-06T02:24:09.521822+0000","last_unstale":"2023-07-06T02:24:10.561486+0000","last_undegraded":"2023-07-06T02:24:10.561486+0000","last_fullsized":"2023-07-06T02:24:10.561486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:24:37.602945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.546638+0000","last_change":"2023-07-06T02:24:10.543291+0000","last_active":"2023-07-06T02:24:10.546638+0000","last_peered":"2023-07-06T02:24:10.546638+0000","last_clean":"2023-07-06T02:24:10.546638+0000","last_became_active":"2023-07-06T02:24:10.543043+0000","last_became_peered":"2023-07-06T02:24:10.543043+0000","last_unstale":"2023-07-06T02:24:10.546638+0000","last_undegraded":"2023-07-06T02:24:10.546638+0000","last_fullsized":"2023-07-06T02:24:10.546638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:59:18.144786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.526259+0000","last_change":"2023-07-06T02:24:09.529211+0000","last_active":"2023-07-06T02:24:10.526259+0000","last_peered":"2023-07-06T02:24:10.526259+0000","last_clean":"2023-07-06T02:24:10.526259+0000","last_became_active":"2023-07-06T02:24:09.528945+0000","last_became_peered":"2023-07-06T02:24:09.528945+0000","last_unstale":"2023-07-06T02:24:10.526259+0000","last_undegraded":"2023-07-06T02:24:10.526259+0000","last_fullsized":"2023-07-06T02:24:10.526259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:32:52.988475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.545463+0000","last_change":"2023-07-06T02:24:10.537932+0000","last_active":"2023-07-06T02:24:10.545463+0000","last_peered":"2023-07-06T02:24:10.545463+0000","last_clean":"2023-07-06T02:24:10.545463+0000","last_became_active":"2023-07-06T02:24:10.537641+0000","last_became_peered":"2023-07-06T02:24:10.537641+0000","last_unstale":"2023-07-06T02:24:10.545463+0000","last_undegraded":"2023-07-06T02:24:10.545463+0000","last_fullsized":"2023-07-06T02:24:10.545463+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:55.365517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.543762+0000","last_change":"2023-07-06T02:24:09.530739+0000","last_active":"2023-07-06T02:24:10.543762+0000","last_peered":"2023-07-06T02:24:10.543762+0000","last_clean":"2023-07-06T02:24:10.543762+0000","last_became_active":"2023-07-06T02:24:09.530478+0000","last_became_peered":"2023-07-06T02:24:09.530478+0000","last_unstale":"2023-07-06T02:24:10.543762+0000","last_undegraded":"2023-07-06T02:24:10.543762+0000","last_fullsized":"2023-07-06T02:24:10.543762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:28.987747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557037+0000","last_change":"2023-07-06T02:24:10.545188+0000","last_active":"2023-07-06T02:24:10.557037+0000","last_peered":"2023-07-06T02:24:10.557037+0000","last_clean":"2023-07-06T02:24:10.557037+0000","last_became_active":"2023-07-06T02:24:10.544930+0000","last_became_peered":"2023-07-06T02:24:10.544930+0000","last_unstale":"2023-07-06T02:24:10.557037+0000","last_undegraded":"2023-07-06T02:24:10.557037+0000","last_fullsized":"2023-07-06T02:24:10.557037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:28.119966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530182+0000","last_change":"2023-07-06T02:24:09.527830+0000","last_active":"2023-07-06T02:24:10.530182+0000","last_peered":"2023-07-06T02:24:10.530182+0000","last_clean":"2023-07-06T02:24:10.530182+0000","last_became_active":"2023-07-06T02:24:09.527543+0000","last_became_peered":"2023-07-06T02:24:09.527543+0000","last_unstale":"2023-07-06T02:24:10.530182+0000","last_undegraded":"2023-07-06T02:24:10.530182+0000","last_fullsized":"2023-07-06T02:24:10.530182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:51.359531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.562038+0000","last_change":"2023-07-06T02:24:10.559205+0000","last_active":"2023-07-06T02:24:10.562038+0000","last_peered":"2023-07-06T02:24:10.562038+0000","last_clean":"2023-07-06T02:24:10.562038+0000","last_became_active":"2023-07-06T02:24:10.558876+0000","last_became_peered":"2023-07-06T02:24:10.558876+0000","last_unstale":"2023-07-06T02:24:10.562038+0000","last_undegraded":"2023-07-06T02:24:10.562038+0000","last_fullsized":"2023-07-06T02:24:10.562038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:12:59.248820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.531433+0000","last_change":"2023-07-06T02:24:09.522250+0000","last_active":"2023-07-06T02:24:10.531433+0000","last_peered":"2023-07-06T02:24:10.531433+0000","last_clean":"2023-07-06T02:24:10.531433+0000","last_became_active":"2023-07-06T02:24:09.521909+0000","last_became_peered":"2023-07-06T02:24:09.521909+0000","last_unstale":"2023-07-06T02:24:10.531433+0000","last_undegraded":"2023-07-06T02:24:10.531433+0000","last_fullsized":"2023-07-06T02:24:10.531433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:59.713337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530186+0000","last_change":"2023-07-06T02:22:47.973696+0000","last_active":"2023-07-06T02:24:10.530186+0000","last_peered":"2023-07-06T02:24:10.530186+0000","last_clean":"2023-07-06T02:24:10.530186+0000","last_became_active":"2023-07-06T02:22:47.973358+0000","last_became_peered":"2023-07-06T02:22:47.973358+0000","last_unstale":"2023-07-06T02:24:10.530186+0000","last_undegraded":"2023-07-06T02:24:10.530186+0000","last_fullsized":"2023-07-06T02:24:10.530186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:34:30.278326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.558866+0000","last_change":"2023-07-06T02:24:10.553865+0000","last_active":"2023-07-06T02:24:10.558866+0000","last_peered":"2023-07-06T02:24:10.558866+0000","last_clean":"2023-07-06T02:24:10.558866+0000","last_became_active":"2023-07-06T02:24:10.553526+0000","last_became_peered":"2023-07-06T02:24:10.553526+0000","last_unstale":"2023-07-06T02:24:10.558866+0000","last_undegraded":"2023-07-06T02:24:10.558866+0000","last_fullsized":"2023-07-06T02:24:10.558866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:06:57.781915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.531751+0000","last_change":"2023-07-06T02:24:09.526273+0000","last_active":"2023-07-06T02:24:10.531751+0000","last_peered":"2023-07-06T02:24:10.531751+0000","last_clean":"2023-07-06T02:24:10.531751+0000","last_became_active":"2023-07-06T02:24:09.525994+0000","last_became_peered":"2023-07-06T02:24:09.525994+0000","last_unstale":"2023-07-06T02:24:10.531751+0000","last_undegraded":"2023-07-06T02:24:10.531751+0000","last_fullsized":"2023-07-06T02:24:10.531751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:02.755463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541431+0000","last_change":"2023-07-06T02:22:47.323237+0000","last_active":"2023-07-06T02:24:10.541431+0000","last_peered":"2023-07-06T02:24:10.541431+0000","last_clean":"2023-07-06T02:24:10.541431+0000","last_became_active":"2023-07-06T02:22:47.322763+0000","last_became_peered":"2023-07-06T02:22:47.322763+0000","last_unstale":"2023-07-06T02:24:10.541431+0000","last_undegraded":"2023-07-06T02:24:10.541431+0000","last_fullsized":"2023-07-06T02:24:10.541431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:32:41.559971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.547054+0000","last_change":"2023-07-06T02:24:10.545773+0000","last_active":"2023-07-06T02:24:10.547054+0000","last_peered":"2023-07-06T02:24:10.547054+0000","last_clean":"2023-07-06T02:24:10.547054+0000","last_became_active":"2023-07-06T02:24:10.545508+0000","last_became_peered":"2023-07-06T02:24:10.545508+0000","last_unstale":"2023-07-06T02:24:10.547054+0000","last_undegraded":"2023-07-06T02:24:10.547054+0000","last_fullsized":"2023-07-06T02:24:10.547054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:29.778784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.540615+0000","last_change":"2023-07-06T02:24:09.525690+0000","last_active":"2023-07-06T02:24:10.540615+0000","last_peered":"2023-07-06T02:24:10.540615+0000","last_clean":"2023-07-06T02:24:10.540615+0000","last_became_active":"2023-07-06T02:24:09.525499+0000","last_became_peered":"2023-07-06T02:24:09.525499+0000","last_unstale":"2023-07-06T02:24:10.540615+0000","last_undegraded":"2023-07-06T02:24:10.540615+0000","last_fullsized":"2023-07-06T02:24:10.540615+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:21:23.653397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.542212+0000","last_change":"2023-07-06T02:22:47.325598+0000","last_active":"2023-07-06T02:24:10.542212+0000","last_peered":"2023-07-06T02:24:10.542212+0000","last_clean":"2023-07-06T02:24:10.542212+0000","last_became_active":"2023-07-06T02:22:47.325315+0000","last_became_peered":"2023-07-06T02:22:47.325315+0000","last_unstale":"2023-07-06T02:24:10.542212+0000","last_undegraded":"2023-07-06T02:24:10.542212+0000","last_fullsized":"2023-07-06T02:24:10.542212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:50:03.777203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530438+0000","last_change":"2023-07-06T02:22:47.326882+0000","last_active":"2023-07-06T02:24:10.530438+0000","last_peered":"2023-07-06T02:24:10.530438+0000","last_clean":"2023-07-06T02:24:10.530438+0000","last_became_active":"2023-07-06T02:22:47.326511+0000","last_became_peered":"2023-07-06T02:22:47.326511+0000","last_unstale":"2023-07-06T02:24:10.530438+0000","last_undegraded":"2023-07-06T02:24:10.530438+0000","last_fullsized":"2023-07-06T02:24:10.530438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:52:30.894947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.560488+0000","last_change":"2023-07-06T02:24:10.553509+0000","last_active":"2023-07-06T02:24:10.560488+0000","last_peered":"2023-07-06T02:24:10.560488+0000","last_clean":"2023-07-06T02:24:10.560488+0000","last_became_active":"2023-07-06T02:24:10.553222+0000","last_became_peered":"2023-07-06T02:24:10.553222+0000","last_unstale":"2023-07-06T02:24:10.560488+0000","last_undegraded":"2023-07-06T02:24:10.560488+0000","last_fullsized":"2023-07-06T02:24:10.560488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:23.099308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.533880+0000","last_change":"2023-07-06T02:24:09.528924+0000","last_active":"2023-07-06T02:24:10.533880+0000","last_peered":"2023-07-06T02:24:10.533880+0000","last_clean":"2023-07-06T02:24:10.533880+0000","last_became_active":"2023-07-06T02:24:09.528612+0000","last_became_peered":"2023-07-06T02:24:09.528612+0000","last_unstale":"2023-07-06T02:24:10.533880+0000","last_undegraded":"2023-07-06T02:24:10.533880+0000","last_fullsized":"2023-07-06T02:24:10.533880+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:16.122139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.529950+0000","last_change":"2023-07-06T02:22:47.613219+0000","last_active":"2023-07-06T02:24:10.529950+0000","last_peered":"2023-07-06T02:24:10.529950+0000","last_clean":"2023-07-06T02:24:10.529950+0000","last_became_active":"2023-07-06T02:22:47.612787+0000","last_became_peered":"2023-07-06T02:22:47.612787+0000","last_unstale":"2023-07-06T02:24:10.529950+0000","last_undegraded":"2023-07-06T02:24:10.529950+0000","last_fullsized":"2023-07-06T02:24:10.529950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:48:56.335387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.557190+0000","last_change":"2023-07-06T02:24:10.545239+0000","last_active":"2023-07-06T02:24:10.557190+0000","last_peered":"2023-07-06T02:24:10.557190+0000","last_clean":"2023-07-06T02:24:10.557190+0000","last_became_active":"2023-07-06T02:24:10.544980+0000","last_became_peered":"2023-07-06T02:24:10.544980+0000","last_unstale":"2023-07-06T02:24:10.557190+0000","last_undegraded":"2023-07-06T02:24:10.557190+0000","last_fullsized":"2023-07-06T02:24:10.557190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:07:17.625774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.559892+0000","last_change":"2023-07-06T02:24:09.524461+0000","last_active":"2023-07-06T02:24:10.559892+0000","last_peered":"2023-07-06T02:24:10.559892+0000","last_clean":"2023-07-06T02:24:10.559892+0000","last_became_active":"2023-07-06T02:24:09.524163+0000","last_became_peered":"2023-07-06T02:24:09.524163+0000","last_unstale":"2023-07-06T02:24:10.559892+0000","last_undegraded":"2023-07-06T02:24:10.559892+0000","last_fullsized":"2023-07-06T02:24:10.559892+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:37:12.190601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.530394+0000","last_change":"2023-07-06T02:22:47.613102+0000","last_active":"2023-07-06T02:24:10.530394+0000","last_peered":"2023-07-06T02:24:10.530394+0000","last_clean":"2023-07-06T02:24:10.530394+0000","last_became_active":"2023-07-06T02:22:47.612655+0000","last_became_peered":"2023-07-06T02:22:47.612655+0000","last_unstale":"2023-07-06T02:24:10.530394+0000","last_undegraded":"2023-07-06T02:24:10.530394+0000","last_fullsized":"2023-07-06T02:24:10.530394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:11:36.055815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555792+0000","last_change":"2023-07-06T02:24:10.553589+0000","last_active":"2023-07-06T02:24:10.555792+0000","last_peered":"2023-07-06T02:24:10.555792+0000","last_clean":"2023-07-06T02:24:10.555792+0000","last_became_active":"2023-07-06T02:24:10.553269+0000","last_became_peered":"2023-07-06T02:24:10.553269+0000","last_unstale":"2023-07-06T02:24:10.555792+0000","last_undegraded":"2023-07-06T02:24:10.555792+0000","last_fullsized":"2023-07-06T02:24:10.555792+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:23:07.130763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.537795+0000","last_change":"2023-07-06T02:24:09.524266+0000","last_active":"2023-07-06T02:24:10.537795+0000","last_peered":"2023-07-06T02:24:10.537795+0000","last_clean":"2023-07-06T02:24:10.537795+0000","last_became_active":"2023-07-06T02:24:09.524003+0000","last_became_peered":"2023-07-06T02:24:09.524003+0000","last_unstale":"2023-07-06T02:24:10.537795+0000","last_undegraded":"2023-07-06T02:24:10.537795+0000","last_fullsized":"2023-07-06T02:24:10.537795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:16.122139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.526759+0000","last_change":"2023-07-06T02:22:47.613009+0000","last_active":"2023-07-06T02:24:10.526759+0000","last_peered":"2023-07-06T02:24:10.526759+0000","last_clean":"2023-07-06T02:24:10.526759+0000","last_became_active":"2023-07-06T02:22:47.612664+0000","last_became_peered":"2023-07-06T02:22:47.612664+0000","last_unstale":"2023-07-06T02:24:10.526759+0000","last_undegraded":"2023-07-06T02:24:10.526759+0000","last_fullsized":"2023-07-06T02:24:10.526759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:33:17.278717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.555076+0000","last_change":"2023-07-06T02:24:10.552095+0000","last_active":"2023-07-06T02:24:10.555076+0000","last_peered":"2023-07-06T02:24:10.555076+0000","last_clean":"2023-07-06T02:24:10.555076+0000","last_became_active":"2023-07-06T02:24:10.551820+0000","last_became_peered":"2023-07-06T02:24:10.551820+0000","last_unstale":"2023-07-06T02:24:10.555076+0000","last_undegraded":"2023-07-06T02:24:10.555076+0000","last_fullsized":"2023-07-06T02:24:10.555076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:45.776434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541893+0000","last_change":"2023-07-06T02:24:09.528276+0000","last_active":"2023-07-06T02:24:10.541893+0000","last_peered":"2023-07-06T02:24:10.541893+0000","last_clean":"2023-07-06T02:24:10.541893+0000","last_became_active":"2023-07-06T02:24:09.527975+0000","last_became_peered":"2023-07-06T02:24:09.527975+0000","last_unstale":"2023-07-06T02:24:10.541893+0000","last_undegraded":"2023-07-06T02:24:10.541893+0000","last_fullsized":"2023-07-06T02:24:10.541893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:39:44.776219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.531349+0000","last_change":"2023-07-06T02:22:47.612949+0000","last_active":"2023-07-06T02:24:10.531349+0000","last_peered":"2023-07-06T02:24:10.531349+0000","last_clean":"2023-07-06T02:24:10.531349+0000","last_became_active":"2023-07-06T02:22:47.612606+0000","last_became_peered":"2023-07-06T02:22:47.612606+0000","last_unstale":"2023-07-06T02:24:10.531349+0000","last_undegraded":"2023-07-06T02:24:10.531349+0000","last_fullsized":"2023-07-06T02:24:10.531349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:38:29.300159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.554722+0000","last_change":"2023-07-06T02:24:10.549495+0000","last_active":"2023-07-06T02:24:10.554722+0000","last_peered":"2023-07-06T02:24:10.554722+0000","last_clean":"2023-07-06T02:24:10.554722+0000","last_became_active":"2023-07-06T02:24:10.549257+0000","last_became_peered":"2023-07-06T02:24:10.549257+0000","last_unstale":"2023-07-06T02:24:10.554722+0000","last_undegraded":"2023-07-06T02:24:10.554722+0000","last_fullsized":"2023-07-06T02:24:10.554722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:09.502630+0000","last_clean_scrub_stamp":"2023-07-06T02:24:09.502630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:43:57.009636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.541602+0000","last_change":"2023-07-06T02:24:09.528277+0000","last_active":"2023-07-06T02:24:10.541602+0000","last_peered":"2023-07-06T02:24:10.541602+0000","last_clean":"2023-07-06T02:24:10.541602+0000","last_became_active":"2023-07-06T02:24:09.527375+0000","last_became_peered":"2023-07-06T02:24:09.527375+0000","last_unstale":"2023-07-06T02:24:10.541602+0000","last_undegraded":"2023-07-06T02:24:10.541602+0000","last_fullsized":"2023-07-06T02:24:10.541602+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:24:08.496619+0000","last_clean_scrub_stamp":"2023-07-06T02:24:08.496619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:43.538847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T02:24:10.536610+0000","last_change":"2023-07-06T02:22:47.329340+0000","last_active":"2023-07-06T02:24:10.536610+0000","last_peered":"2023-07-06T02:24:10.536610+0000","last_clean":"2023-07-06T02:24:10.536610+0000","last_became_active":"2023-07-06T02:22:47.329090+0000","last_became_peered":"2023-07-06T02:22:47.329090+0000","last_unstale":"2023-07-06T02:24:10.536610+0000","last_undegraded":"2023-07-06T02:24:10.536610+0000","last_fullsized":"2023-07-06T02:24:10.536610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T02:22:46.309625+0000","last_clean_scrub_stamp":"2023-07-06T02:22:46.309625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:43:10.456578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":1024000,"data_stored":781094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.501}]},{"osd":1,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.372},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.45600000000000002}]},{"osd":5,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":434176,"data_stored":191308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.623},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.435},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.46500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":1024000,"data_stored":781074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.375}]},{"osd":3,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.755}]},{"osd":6,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.76700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":409600,"data_stored":181827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.317},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.27200000000000002}]},{"osd":3,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.372},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.504},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.442},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":192222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.377}]},{"osd":3,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jul 6 02:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":183188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.255},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.317}]},{"osd":4,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.497}]},{"osd":7,"last update":"Thu Jul 6 02:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":183188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jul 6 02:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":183188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.623},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jul 6 02:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T02:24:15.020 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T02:24:15.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T02:24:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.178+0000 7f387ce48700 1 Processor -- start 2023-07-06T02:24:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.178+0000 7f387ce48700 1 -- start start 2023-07-06T02:24:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387ce48700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3878156ab0 0x7f3878160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387ce48700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38780cb3d0 con 0x7f3878160bb0 2023-07-06T02:24:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387ce48700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3878156e80 con 0x7f38781600b0 2023-07-06T02:24:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387ce48700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38780ca370 con 0x7f3878156ab0 2023-07-06T02:24:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f3875d9b700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3878160bb0 0x7f38780b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59728/0 (socket says 172.21.15.89:59728) 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f3875d9b700 1 -- 172.21.15.89:0/2828060101 learned_addr learned my addr 172.21.15.89:0/2828060101 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387659c700 1 --2- 172.21.15.89:0/2828060101 >> v2:172.21.15.172:3300/0 conn(0x7f3878156ab0 0x7f3878160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662694040 0 0) 0x7f38780cb3d0 con 0x7f3878160bb0 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38600035b0 con 0x7f3878160bb0 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1439015098 0 0) 0x7f38600035b0 con 0x7f3878160bb0 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 >> v2:172.21.15.172:3300/0 conn(0x7f3878156ab0 msgr2=0x7f3878160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 --2- 172.21.15.89:0/2828060101 >> v2:172.21.15.172:3300/0 conn(0x7f3878156ab0 0x7f3878160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 >> v1:172.21.15.172:6789/0 conn(0x7f38781600b0 legacy=0x7f38780cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38780ca610 con 0x7f3878160bb0 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f386c001670 con 0x7f3878160bb0 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.179+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 >> v1:172.21.15.89:6789/0 conn(0x7f3878160bb0 legacy=0x7f38780b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3860003a00 0x7f3860003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860004850 0x7f3860004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3860005200 0x7f386000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38780ca370 con 0x7f3860003a00 2023-07-06T02:24:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386000c080 con 0x7f3860005200 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386000c240 con 0x7f3860004850 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387659c700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3860003a00 0x7f3860003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f3876d9d700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3860005200 0x7f386000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f3875d9b700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860004850 0x7f3860004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387659c700 1 -- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860004850 msgr2=0x7f3860004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387659c700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860004850 0x7f3860004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387659c700 1 -- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3860005200 msgr2=0x7f386000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387659c700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3860005200 0x7f386000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387659c700 1 -- 172.21.15.89:0/2828060101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600035b0 con 0x7f3860003a00 2023-07-06T02:24:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f3876d9d700 1 -- 172.21.15.89:0/2828060101 reap_dead start 2023-07-06T02:24:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387659c700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3860003a00 0x7f3860003e10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3868009b60 tx=0x7f3860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8572823c2be7ea0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386800e6c0 con 0x7f3860003a00 2023-07-06T02:24:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386800e820 con 0x7f3860003a00 2023-07-06T02:24:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.180+0000 7f387559a700 1 -- 172.21.15.89:0/2828060101 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386800eb90 con 0x7f3860003a00 2023-07-06T02:24:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.181+0000 7f387ce48700 1 -- 172.21.15.89:0/2828060101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3860003a00 msgr2=0x7f3860003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.181+0000 7f387ce48700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3860003a00 0x7f3860003e10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3868009b60 tx=0x7f3860005880 comp rx=0 tx=0).stop 2023-07-06T02:24:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.181+0000 7f387ce48700 1 -- 172.21.15.89:0/2828060101 shutdown_connections 2023-07-06T02:24:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.181+0000 7f387ce48700 1 --2- 172.21.15.89:0/2828060101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3860003a00 0x7f3860003e10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.181+0000 7f387ce48700 1 -- 172.21.15.89:0/2828060101 >> 172.21.15.89:0/2828060101 conn(0x7f38780cc490 msgr2=0x7f38780cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.181+0000 7f387ce48700 1 -- 172.21.15.89:0/2828060101 shutdown_connections 2023-07-06T02:24:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.181+0000 7f387ce48700 1 -- 172.21.15.89:0/2828060101 wait complete. 2023-07-06T02:24:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.182+0000 7f387ce48700 1 Processor -- start 2023-07-06T02:24:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.182+0000 7f387ce48700 1 -- start start 2023-07-06T02:24:15.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f387ce48700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860003a00 0x7f38780b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f387ce48700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38781e8880 0x7f3878142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f387ce48700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3878142e80 0x7f38780ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f387ce48700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38780c6310 con 0x7f3878142e80 2023-07-06T02:24:15.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f387ce48700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38780b5270 con 0x7f38781e8880 2023-07-06T02:24:15.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f387ce48700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38780b48a0 con 0x7f3860003a00 2023-07-06T02:24:15.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3878142e80 0x7f38780ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3878142e80 0x7f38780ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34840/0 (socket says 172.21.15.89:34840) 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 -- 172.21.15.89:0/2774146363 learned_addr learned my addr 172.21.15.89:0/2774146363 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 -- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860003a00 msgr2=0x7f38780b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3875d9b700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38781e8880 0x7f3878142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f387659c700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860003a00 0x7f38780b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860003a00 0x7f38780b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 -- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38781e8880 msgr2=0x7f3878142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38781e8880 0x7f3878142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 -- 172.21.15.89:0/2774146363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38780b3c90 con 0x7f3878142e80 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3875d9b700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38781e8880 0x7f3878142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.183+0000 7f3876d9d700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3878142e80 0x7f38780ad680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f387806e990 tx=0x7f387000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.184+0000 7f3866ffd700 1 -- 172.21.15.89:0/2774146363 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870003cd0 con 0x7f3878142e80 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.184+0000 7f3866ffd700 1 -- 172.21.15.89:0/2774146363 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870003e30 con 0x7f3878142e80 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.184+0000 7f3866ffd700 1 -- 172.21.15.89:0/2774146363 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387000da10 con 0x7f3878142e80 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.184+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38780b3f70 con 0x7f3878142e80 2023-07-06T02:24:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.184+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38780b4530 con 0x7f3878142e80 2023-07-06T02:24:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.185+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3878074270 con 0x7f3878142e80 2023-07-06T02:24:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.185+0000 7f3866ffd700 1 -- 172.21.15.89:0/2774146363 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f387000db70 con 0x7f3878142e80 2023-07-06T02:24:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.185+0000 7f3866ffd700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f385806ace0 0x7f385806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.186+0000 7f3866ffd700 1 -- 172.21.15.89:0/2774146363 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3870089770 con 0x7f3878142e80 2023-07-06T02:24:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.186+0000 7f387659c700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f385806ace0 0x7f385806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.186+0000 7f387659c700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f385806ace0 0x7f385806d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f38680090b0 tx=0x7f3868008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.190+0000 7f3866ffd700 1 -- 172.21.15.89:0/2774146363 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3870054770 con 0x7f3878142e80 2023-07-06T02:24:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.455+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f38780b49e0 con 0x7f3878142e80 2023-07-06T02:24:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.459+0000 7f3866ffd700 1 -- 172.21.15.89:0/2774146363 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3870057d90 con 0x7f3878142e80 2023-07-06T02:24:15.462 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2023-07-06T02:24:15.463 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBJJaZkfY62AxAAWC7BJI+7fFi+ew8E4NJVSA== 2023-07-06T02:24:15.463 INFO:teuthology.orchestra.run.smithi089.stdout: caps mds = "allow" 2023-07-06T02:24:15.463 INFO:teuthology.orchestra.run.smithi089.stdout: caps mon = "allow r" 2023-07-06T02:24:15.463 INFO:teuthology.orchestra.run.smithi089.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T02:24:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f385806ace0 msgr2=0x7f385806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f385806ace0 0x7f385806d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f38680090b0 tx=0x7f3868008030 comp rx=0 tx=0).stop 2023-07-06T02:24:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3878142e80 msgr2=0x7f38780ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3878142e80 0x7f38780ad680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f387806e990 tx=0x7f387000d150 comp rx=0 tx=0).stop 2023-07-06T02:24:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 shutdown_connections 2023-07-06T02:24:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f385806ace0 0x7f385806d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3878142e80 0x7f38780ad680 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3860003a00 0x7f38780b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 --2- 172.21.15.89:0/2774146363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38781e8880 0x7f3878142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 >> 172.21.15.89:0/2774146363 conn(0x7f38780cc490 msgr2=0x7f38780ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 shutdown_connections 2023-07-06T02:24:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.462+0000 7f387ce48700 1 -- 172.21.15.89:0/2774146363 wait complete. 2023-07-06T02:24:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-07-06T02:24:15.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T02:24:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.595+0000 7fa3c7902700 1 Processor -- start 2023-07-06T02:24:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.595+0000 7fa3c7902700 1 -- start start 2023-07-06T02:24:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c7902700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3c0156ab0 0x7fa3c0160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c7902700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3c0160bb0 0x7fa3c00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c7902700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c01600b0 0x7fa3c00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c7902700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c00cb3d0 con 0x7fa3c0156ab0 2023-07-06T02:24:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c7902700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c0156e80 con 0x7fa3c0160bb0 2023-07-06T02:24:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c7902700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00ca370 con 0x7fa3c01600b0 2023-07-06T02:24:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c569e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3c0156ab0 0x7fa3c0160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c569e700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa3c0156ab0 0x7fa3c0160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34844/0 (socket says 172.21.15.89:34844) 2023-07-06T02:24:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c569e700 1 -- 172.21.15.89:0/2852399286 learned_addr learned my addr 172.21.15.89:0/2852399286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c4e9d700 1 --2- 172.21.15.89:0/2852399286 >> v2:172.21.15.172:3300/0 conn(0x7fa3c0160bb0 0x7fa3c00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.596+0000 7fa3c5e9f700 1 --2- 172.21.15.89:0/2852399286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c01600b0 0x7fa3c00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c569e700 1 -- 172.21.15.89:0/2852399286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c01600b0 msgr2=0x7fa3c00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c569e700 1 --2- 172.21.15.89:0/2852399286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c01600b0 0x7fa3c00b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c569e700 1 -- 172.21.15.89:0/2852399286 >> v2:172.21.15.172:3300/0 conn(0x7fa3c0160bb0 msgr2=0x7fa3c00b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c569e700 1 --2- 172.21.15.89:0/2852399286 >> v2:172.21.15.172:3300/0 conn(0x7fa3c0160bb0 0x7fa3c00b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c569e700 1 -- 172.21.15.89:0/2852399286 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00ca610 con 0x7fa3c0156ab0 2023-07-06T02:24:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c569e700 1 --2- 172.21.15.89:0/2852399286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0156ab0 0x7fa3c0160670 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00c6a70 tx=0x7fa3b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c6441bb82c7594 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3b7fff700 1 -- 172.21.15.89:0/2852399286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000b870 con 0x7fa3c0156ab0 2023-07-06T02:24:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3b7fff700 1 -- 172.21.15.89:0/2852399286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000b9d0 con 0x7fa3c0156ab0 2023-07-06T02:24:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3b7fff700 1 -- 172.21.15.89:0/2852399286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000bd60 con 0x7fa3c0156ab0 2023-07-06T02:24:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c7902700 1 -- 172.21.15.89:0/2852399286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0156ab0 msgr2=0x7fa3c0160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2852399286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0156ab0 0x7fa3c0160670 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00c6a70 tx=0x7fa3b0009bd0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c7902700 1 -- 172.21.15.89:0/2852399286 shutdown_connections 2023-07-06T02:24:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2852399286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c01600b0 0x7fa3c00b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2852399286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0156ab0 0x7fa3c0160670 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2852399286 >> v2:172.21.15.172:3300/0 conn(0x7fa3c0160bb0 0x7fa3c00b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.597+0000 7fa3c7902700 1 -- 172.21.15.89:0/2852399286 >> 172.21.15.89:0/2852399286 conn(0x7fa3c00cc490 msgr2=0x7fa3c00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.598+0000 7fa3c7902700 1 -- 172.21.15.89:0/2852399286 shutdown_connections 2023-07-06T02:24:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.598+0000 7fa3c7902700 1 -- 172.21.15.89:0/2852399286 wait complete. 2023-07-06T02:24:15.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 Processor -- start 2023-07-06T02:24:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 -- start start 2023-07-06T02:24:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c0156ab0 0x7fa3c0153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c01600b0 0x7fa3c01e8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00b51c0 0x7fa3c006bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01e89a0 con 0x7fa3c00b51c0 2023-07-06T02:24:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01e73c0 con 0x7fa3c01600b0 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.599+0000 7fa3c7902700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01e76a0 con 0x7fa3c0156ab0 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00b51c0 0x7fa3c006bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00b51c0 0x7fa3c006bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34854/0 (socket says 172.21.15.89:34854) 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 -- 172.21.15.89:0/2729433688 learned_addr learned my addr 172.21.15.89:0/2729433688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c4e9d700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c01600b0 0x7fa3c01e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c569e700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c0156ab0 0x7fa3c0153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 -- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c0156ab0 msgr2=0x7fa3c0153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c0156ab0 0x7fa3c0153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 -- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c01600b0 msgr2=0x7fa3c01e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c01600b0 0x7fa3c01e8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 -- 172.21.15.89:0/2729433688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0009550 con 0x7fa3c00b51c0 2023-07-06T02:24:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c5e9f700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00b51c0 0x7fa3c006bd70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8007e30 tx=0x7fa3b800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3b5ffb700 1 -- 172.21.15.89:0/2729433688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800ed30 con 0x7fa3c00b51c0 2023-07-06T02:24:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3b5ffb700 1 -- 172.21.15.89:0/2729433688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b800ee90 con 0x7fa3c00b51c0 2023-07-06T02:24:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c01e7980 con 0x7fa3c00b51c0 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.600+0000 7fa3b5ffb700 1 -- 172.21.15.89:0/2729433688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8014660 con 0x7fa3c00b51c0 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.601+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c01e7f40 con 0x7fa3c00b51c0 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.601+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c0074270 con 0x7fa3c00b51c0 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.602+0000 7fa3b5ffb700 1 -- 172.21.15.89:0/2729433688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3b80147c0 con 0x7fa3c00b51c0 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.602+0000 7fa3b5ffb700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3a406ace0 0x7fa3a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.602+0000 7fa3c569e700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3a406ace0 0x7fa3a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.602+0000 7fa3b5ffb700 1 -- 172.21.15.89:0/2729433688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa3b8089da0 con 0x7fa3c00b51c0 2023-07-06T02:24:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.603+0000 7fa3c569e700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3a406ace0 0x7fa3a406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000c00 tx=0x7fa3b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.607+0000 7fa3b5ffb700 1 -- 172.21.15.89:0/2729433688 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3b8058670 con 0x7fa3c00b51c0 2023-07-06T02:24:15.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.838+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3c015e350 con 0x7fa3c00b51c0 2023-07-06T02:24:15.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.838+0000 7fa3b5ffb700 1 -- 172.21.15.89:0/2729433688 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4668 (secure 0 0 0) 0x7fa3b8058200 con 0x7fa3c00b51c0 2023-07-06T02:24:15.842 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:15.842 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/34790451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":34790451},{"type":"v1","addr":"172.21.15.172:6835","nonce":34790451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3141331417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3141331417},{"type":"v1","addr":"172.21.15.172:6834","nonce":3141331417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6836/2965598582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2965598582},{"type":"v1","addr":"172.21.15.89:6836","nonce":2965598582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T02:24:09.907406+0000","modified":"2023-07-06T02:24:13.543375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5469":{"gid":5469,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6837/326075892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6835","nonce":326075892},{"type":"v1","addr":"172.21.15.89:6837","nonce":326075892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T02:24:15.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3a406ace0 msgr2=0x7fa3a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3a406ace0 0x7fa3a406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000c00 tx=0x7fa3b0008030 comp rx=0 tx=0).stop 2023-07-06T02:24:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00b51c0 msgr2=0x7fa3c006bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00b51c0 0x7fa3c006bd70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8007e30 tx=0x7fa3b800e3c0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 shutdown_connections 2023-07-06T02:24:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fa3a406ace0 0x7fa3a406d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00b51c0 0x7fa3c006bd70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c0156ab0 0x7fa3c0153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 --2- 172.21.15.89:0/2729433688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c01600b0 0x7fa3c01e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.840+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 >> 172.21.15.89:0/2729433688 conn(0x7fa3c00cc490 msgr2=0x7fa3c0154780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.841+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 shutdown_connections 2023-07-06T02:24:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:15.841+0000 7fa3c7902700 1 -- 172.21.15.89:0/2729433688 wait complete. 2023-07-06T02:24:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2023-07-06T02:24:16.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T02:24:09.907406+0000', 'modified': '2023-07-06T02:24:13.543375+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5469': {'gid': 5469, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6837/326075892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6835', 'nonce': 326075892}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 326075892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T02:24:16.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T02:24:16.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T02:24:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.380+0000 7f51b99eb700 1 Processor -- start 2023-07-06T02:24:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.380+0000 7f51b99eb700 1 -- start start 2023-07-06T02:24:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b99eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b415c4f0 0x7f51b40b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b99eb700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51b4070920 0x7f51b40cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b99eb700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b40b5960 con 0x7f51b40ccfe0 2023-07-06T02:24:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b99eb700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b40cf4a0 con 0x7f51b4070920 2023-07-06T02:24:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b99eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40b38d0 con 0x7f51b415c4f0 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b37fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51b4070920 0x7f51b40cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b27fc700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f51b40ccfe0 0x7f51b4070210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59762/0 (socket says 172.21.15.89:59762) 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b27fc700 1 -- 172.21.15.89:0/2341826447 learned_addr learned my addr 172.21.15.89:0/2341826447 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/2341826447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b415c4f0 0x7f51b40b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b37fe700 1 -- 172.21.15.89:0/2341826447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b415c4f0 msgr2=0x7f51b40b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b37fe700 1 --2- 172.21.15.89:0/2341826447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b415c4f0 0x7f51b40b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b37fe700 1 -- 172.21.15.89:0/2341826447 >> v1:172.21.15.89:6789/0 conn(0x7f51b40ccfe0 legacy=0x7f51b4070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b37fe700 1 -- 172.21.15.89:0/2341826447 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b40b3bb0 con 0x7f51b4070920 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b1ffb700 1 -- 172.21.15.89:0/2341826447 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449177196 0 0) 0x7f51b40b5960 con 0x7f51b40ccfe0 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/2341826447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b415c4f0 0x7f51b40b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b37fe700 1 --2- 172.21.15.89:0/2341826447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40cb5f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009d90 tx=0x7f51ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9fcd8fc266d5b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b1ffb700 1 -- 172.21.15.89:0/2341826447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00e070 con 0x7f51b4070920 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b1ffb700 1 -- 172.21.15.89:0/2341826447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac00bbd0 con 0x7f51b4070920 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.381+0000 7f51b1ffb700 1 -- 172.21.15.89:0/2341826447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac012430 con 0x7f51b4070920 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 -- 172.21.15.89:0/2341826447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 msgr2=0x7f51b40cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 --2- 172.21.15.89:0/2341826447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40cb5f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009d90 tx=0x7f51ac00b330 comp rx=0 tx=0).stop 2023-07-06T02:24:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 -- 172.21.15.89:0/2341826447 shutdown_connections 2023-07-06T02:24:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 --2- 172.21.15.89:0/2341826447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40cb5f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 --2- 172.21.15.89:0/2341826447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b415c4f0 0x7f51b40b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 -- 172.21.15.89:0/2341826447 >> 172.21.15.89:0/2341826447 conn(0x7f51b40d09e0 msgr2=0x7f51b40d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 -- 172.21.15.89:0/2341826447 shutdown_connections 2023-07-06T02:24:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.382+0000 7f51b99eb700 1 -- 172.21.15.89:0/2341826447 wait complete. 2023-07-06T02:24:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.383+0000 7f51b99eb700 1 Processor -- start 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.383+0000 7f51b99eb700 1 -- start start 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b99eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b99eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b40ccfe0 0x7f51b406dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b99eb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51b415c4f0 0x7f51b41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b99eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b406dff0 con 0x7f51b4070920 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b99eb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40c8140 con 0x7f51b415c4f0 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b99eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40c8420 con 0x7f51b40ccfe0 2023-07-06T02:24:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34864/0 (socket says 172.21.15.89:34864) 2023-07-06T02:24:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1452139342 learned_addr learned my addr 172.21.15.89:0/1452139342 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b37fe700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51b415c4f0 0x7f51b41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b27fc700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b40ccfe0 0x7f51b406dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b40ccfe0 msgr2=0x7f51b406dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b40ccfe0 0x7f51b406dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51b415c4f0 msgr2=0x7f51b41545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51b415c4f0 0x7f51b41545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1452139342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac009a70 con 0x7f51b4070920 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.384+0000 7f51b37fe700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51b415c4f0 0x7f51b41545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.385+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40c7ab0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c7f0 tx=0x7f51a400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.385+0000 7f519bfff700 1 -- 172.21.15.89:0/1452139342 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4011800 con 0x7f51b4070920 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.385+0000 7f519bfff700 1 -- 172.21.15.89:0/1452139342 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a4011960 con 0x7f51b4070920 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.385+0000 7f519bfff700 1 -- 172.21.15.89:0/1452139342 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4011d10 con 0x7f51b4070920 2023-07-06T02:24:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.385+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b40c8700 con 0x7f51b4070920 2023-07-06T02:24:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.385+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b40c8cc0 con 0x7f51b4070920 2023-07-06T02:24:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.386+0000 7f519bfff700 1 -- 172.21.15.89:0/1452139342 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51a4018360 con 0x7f51b4070920 2023-07-06T02:24:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.387+0000 7f519bfff700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51a006ace0 0x7f51a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.387+0000 7f519bfff700 1 -- 172.21.15.89:0/1452139342 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f51a4089510 con 0x7f51b4070920 2023-07-06T02:24:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.387+0000 7f51b27fc700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51a006ace0 0x7f51a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.387+0000 7f51b27fc700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51a006ace0 0x7f51a006d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51b40b4bf0 tx=0x7f51a8007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.388+0000 7f51c5b70b80 1 Processor -- start 2023-07-06T02:24:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.388+0000 7f51c5b70b80 1 -- start start 2023-07-06T02:24:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccc470 0x559fd4ceb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4ccf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccccd0 0x559fd4ccf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4ba9c60 con 0x559fd4ccc470 2023-07-06T02:24:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4bfc9f0 con 0x559fd4ccccd0 2023-07-06T02:24:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fd4bf5e80 con 0x559fd4ca2830 2023-07-06T02:24:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51b2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccc470 0x559fd4ceb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51b2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccc470 0x559fd4ceb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34872/0 (socket says 172.21.15.89:34872) 2023-07-06T02:24:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.389+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1216321617 learned_addr learned my addr 172.21.15.89:0/1216321617 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1216321617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 msgr2=0x559fd4ccf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4ccf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1216321617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccccd0 msgr2=0x559fd4ccf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51b37fe700 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccccd0 0x559fd4ccf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccccd0 0x559fd4ccf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51b2ffd700 1 -- 172.21.15.89:0/1216321617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd4a23d90 con 0x559fd4ccc470 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccc470 0x559fd4ceb670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f51a405fff0 tx=0x7f51a40605a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5be9882515deeb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51997fa700 1 -- 172.21.15.89:0/1216321617 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4060090 con 0x559fd4ccc470 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51997fa700 1 -- 172.21.15.89:0/1216321617 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a40601f0 con 0x559fd4ccc470 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.390+0000 7f51997fa700 1 -- 172.21.15.89:0/1216321617 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4060360 con 0x559fd4ccc470 2023-07-06T02:24:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 -- 172.21.15.89:0/1216321617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccc470 msgr2=0x559fd4ceb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccc470 0x559fd4ceb670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f51a405fff0 tx=0x7f51a40605a0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 -- 172.21.15.89:0/1216321617 shutdown_connections 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccccd0 0x559fd4ccf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccc470 0x559fd4ceb670 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/1216321617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4ccf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 -- 172.21.15.89:0/1216321617 >> 172.21.15.89:0/1216321617 conn(0x559fd4c99e80 msgr2=0x559fd4cea1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 -- 172.21.15.89:0/1216321617 shutdown_connections 2023-07-06T02:24:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 -- 172.21.15.89:0/1216321617 wait complete. 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 -- start start 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.391+0000 7f51c5b70b80 1 Processor -- start 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4cc8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4ca4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4b95a70 con 0x559fd4ccccd0 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4b95770 con 0x559fd4ccc470 2023-07-06T02:24:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fd4a798d0 con 0x559fd4ca2830 2023-07-06T02:24:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51c5b70b80 10 client.0 _ll_register_callbacks cb 0x559fd4b93710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T02:24:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34882/0 (socket says 172.21.15.89:34882) 2023-07-06T02:24:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.392+0000 7f51b37fe700 1 -- 172.21.15.89:0/3709980505 learned_addr learned my addr 172.21.15.89:0/3709980505 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b27fc700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4ca4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b37fe700 1 -- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 msgr2=0x559fd4cc8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4cc8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4cc8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b37fe700 1 -- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 msgr2=0x559fd4ca4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4ca4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b37fe700 1 -- 172.21.15.89:0/3709980505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a4011e70 con 0x559fd4ccccd0 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b27fc700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4ca4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca4640 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f51b40d1980 tx=0x7f51ac004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d8f59002616ab3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51927fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-07-06T02:24:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51927fc700 1 -- 172.21.15.89:0/3709980505 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac003e20 con 0x559fd4ccccd0 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fd4c97490 con 0x559fd4ccccd0 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51927fc700 1 -- 172.21.15.89:0/3709980505 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f51ac012d80 con 0x559fd4ccccd0 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51927fc700 1 -- 172.21.15.89:0/3709980505 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51ac004040 con 0x559fd4ccccd0 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 10 client.5661 fetch_fsmap learned FSMap version 17 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x559fd4a0cc10 con 0x559fd4ccccd0 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51927fc700 1 -- 172.21.15.89:0/3709980505 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559fd4c97490 con 0x559fd4ccccd0 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51927fc700 1 -- 172.21.15.89:0/3709980505 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f51ac0041a0 con 0x559fd4ccccd0 2023-07-06T02:24:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 10 client.5661 fetch_fsmap finished waiting for FSMap version 17 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 10 client.5661 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 20 client.5661 populate_metadata read hostname 'smithi089' 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4cd1a60 0x559fd4cec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 4 client.5661 mds_command: new command op to 5469 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 --> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fd4beee60 con 0x559fd4cd1a60 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.393+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4cd1a60 0x559fd4cec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51927fc700 10 client.5661 ms_handle_connect on v2:172.21.15.89:6835/326075892 2023-07-06T02:24:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4cd1a60 0x559fd4cec260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=852b45e4a34875b1 server_cookie=2f1fd7e5eaafce1f in_seq=0 out_seq=0 2023-07-06T02:24:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51927fc700 1 -- 172.21.15.89:0/3709980505 <== mds.0 v2:172.21.15.89:6835/326075892 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559fd4beee60 con 0x559fd4cd1a60 2023-07-06T02:24:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51927fc700 10 client.5661 handle_command_reply: tid=0 2023-07-06T02:24:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 client.5661 shutdown 2023-07-06T02:24:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 msgr2=0x559fd4ca4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca4640 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f51b40d1980 tx=0x7f51ac004b40 comp rx=0 tx=0).stop 2023-07-06T02:24:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 shutdown_connections 2023-07-06T02:24:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4cd1a60 0x559fd4cec260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca4640 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4cc8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3709980505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4ca4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51927fc700 0 client.5661 ms_handle_reset on v2:172.21.15.89:6835/326075892 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.394+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 >> 172.21.15.89:0/3709980505 conn(0x559fd4c99e80 msgr2=0x559fd4c9fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.395+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 shutdown_connections 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.395+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3709980505 wait complete. 2023-07-06T02:24:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.395+0000 7f51c5b70b80 20 client.5661 trim_cache size 0 max 16384 2023-07-06T02:24:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.412+0000 7f51c5b70b80 1 Processor -- start 2023-07-06T02:24:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.412+0000 7f51c5b70b80 1 -- start start 2023-07-06T02:24:16.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4ca42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4cdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ce00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4cd0ec0 con 0x559fd4ccccd0 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4ca51c0 con 0x559fd4ccc470 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fd4ce26f0 con 0x559fd4ca2830 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ce00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ce00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34888/0 (socket says 172.21.15.89:34888) 2023-07-06T02:24:16.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 -- 172.21.15.89:0/3524911548 learned_addr learned my addr 172.21.15.89:0/3524911548 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b27fc700 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4cdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4ca42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 -- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 msgr2=0x559fd4ca42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4ca42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 -- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 msgr2=0x559fd4cdfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4cdfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 -- 172.21.15.89:0/3524911548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd4a4a200 con 0x559fd4ccccd0 2023-07-06T02:24:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.413+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ce00c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0125b0 tx=0x7f51ac01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ba4d3e096163cf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f5199ffb700 1 -- 172.21.15.89:0/3524911548 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac004660 con 0x559fd4ccccd0 2023-07-06T02:24:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f5199ffb700 1 -- 172.21.15.89:0/3524911548 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac004040 con 0x559fd4ccccd0 2023-07-06T02:24:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f5199ffb700 1 -- 172.21.15.89:0/3524911548 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac032440 con 0x559fd4ccccd0 2023-07-06T02:24:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3524911548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 msgr2=0x559fd4ce00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ce00c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0125b0 tx=0x7f51ac01e040 comp rx=0 tx=0).stop 2023-07-06T02:24:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3524911548 shutdown_connections 2023-07-06T02:24:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ce00c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ca2830 0x559fd4ca42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3524911548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ccc470 0x559fd4cdfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3524911548 >> 172.21.15.89:0/3524911548 conn(0x559fd4c99e80 msgr2=0x559fd4ca1750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3524911548 shutdown_connections 2023-07-06T02:24:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.414+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3524911548 wait complete. 2023-07-06T02:24:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.415+0000 7f51c5b70b80 1 -- start start 2023-07-06T02:24:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.415+0000 7f51c5b70b80 1 Processor -- start 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ca2830 0x559fd4d39aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ccc470 0x559fd4cec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4d51840 con 0x559fd4ccccd0 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd4ced000 con 0x559fd4ca2830 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fd4ced250 con 0x559fd4ccc470 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 10 client.0 _ll_register_callbacks cb 0x7f51ac003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T02:24:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34896/0 (socket says 172.21.15.89:34896) 2023-07-06T02:24:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 -- 172.21.15.89:0/3535393911 learned_addr learned my addr 172.21.15.89:0/3535393911 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b27fc700 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ccc470 0x559fd4cec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ca2830 0x559fd4d39aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 -- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ccc470 msgr2=0x559fd4cec520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ccc470 0x559fd4cec520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 -- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ca2830 msgr2=0x559fd4d39aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ca2830 0x559fd4d39aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 -- 172.21.15.89:0/3535393911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fd4a0cc10 con 0x559fd4ccccd0 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51b37fe700 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca5550 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009d90 tx=0x7f51ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91c612807ee58a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51927fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51927fc700 1 -- 172.21.15.89:0/3535393911 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00e040 con 0x559fd4ccccd0 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51927fc700 1 -- 172.21.15.89:0/3535393911 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f51ac004660 con 0x559fd4ccccd0 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.416+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51a40601f0 con 0x559fd4ccccd0 2023-07-06T02:24:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51927fc700 1 -- 172.21.15.89:0/3535393911 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51ac003ec0 con 0x559fd4ccccd0 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 10 client.5667 fetch_fsmap learned FSMap version 17 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x559fd4a22260 con 0x559fd4ccccd0 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51927fc700 1 -- 172.21.15.89:0/3535393911 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a40601f0 con 0x559fd4ccccd0 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51927fc700 1 -- 172.21.15.89:0/3535393911 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f51ac03a020 con 0x559fd4ccccd0 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 10 client.5667 fetch_fsmap finished waiting for FSMap version 17 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 10 client.5667 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 20 client.5667 populate_metadata read hostname 'smithi089' 2023-07-06T02:24:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4d507f0 0x559fd4d50bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 4 client.5667 mds_command: new command op to 5469 tid=0[{"prefix": "status"}] 2023-07-06T02:24:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 --> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559fd4bb1000 con 0x559fd4d507f0 2023-07-06T02:24:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4d507f0 0x559fd4d50bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51927fc700 10 client.5667 ms_handle_connect on v2:172.21.15.89:6835/326075892 2023-07-06T02:24:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51b2ffd700 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4d507f0 0x559fd4d50bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43eec62dd26fddee server_cookie=30c9b7e45369cae5 in_seq=0 out_seq=0 2023-07-06T02:24:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51927fc700 1 -- 172.21.15.89:0/3535393911 <== mds.0 v2:172.21.15.89:6835/326075892 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559fd4bb1000 con 0x559fd4d507f0 2023-07-06T02:24:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51927fc700 10 client.5667 handle_command_reply: tid=0 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 1 client.5667 shutdown 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 msgr2=0x559fd4ca5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.417+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca5550 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009d90 tx=0x7f51ac009750 comp rx=0 tx=0).stop 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 shutdown_connections 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:6835/326075892,v1:172.21.15.89:6837/326075892] conn(0x559fd4d507f0 0x559fd4d50bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559fd4ccccd0 0x559fd4ca5550 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559fd4ca2830 0x559fd4d39aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51927fc700 0 client.5667 ms_handle_reset on v2:172.21.15.89:6835/326075892 2023-07-06T02:24:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 --2- 172.21.15.89:0/3535393911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559fd4ccc470 0x559fd4cec520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 >> 172.21.15.89:0/3535393911 conn(0x559fd4c99e80 msgr2=0x559fd4ccb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 shutdown_connections 2023-07-06T02:24:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 1 -- 172.21.15.89:0/3535393911 wait complete. 2023-07-06T02:24:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51c5b70b80 20 client.5667 trim_cache size 0 max 16384 2023-07-06T02:24:16.430 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "b61ed025-dcde-49f5-a87d-8904066cfa54", 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 5469, 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 5.8962669209999996, 2023-07-06T02:24:16.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 17, 2023-07-06T02:24:16.432 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 38, 2023-07-06T02:24:16.432 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 38, 2023-07-06T02:24:16.432 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 13.789510697000001 2023-07-06T02:24:16.432 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-07-06T02:24:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51a006ace0 msgr2=0x7f51a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51a006ace0 0x7f51a006d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51b40b4bf0 tx=0x7f51a8007be0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 msgr2=0x7f51b40c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40c7ab0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c7f0 tx=0x7f51a400cbb0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 shutdown_connections 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51b415c4f0 0x7f51b41545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f51a006ace0 0x7f51a006d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4070920 0x7f51b40c7ab0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 --2- 172.21.15.89:0/1452139342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51b40ccfe0 0x7f51b406dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 >> 172.21.15.89:0/1452139342 conn(0x7f51b40d09e0 msgr2=0x7f51b4159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 shutdown_connections 2023-07-06T02:24:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.418+0000 7f51b99eb700 1 -- 172.21.15.89:0/1452139342 wait complete. 2023-07-06T02:24:16.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T02:24:16.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.552+0000 7f45df8a1700 1 Processor -- start 2023-07-06T02:24:16.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.553+0000 7f45df8a1700 1 -- start start 2023-07-06T02:24:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.553+0000 7f45df8a1700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f45d8156ab0 0x7f45d8160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.553+0000 7f45df8a1700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d80cb3d0 con 0x7f45d8160bb0 2023-07-06T02:24:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.553+0000 7f45df8a1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d8156e80 con 0x7f45d81600b0 2023-07-06T02:24:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.553+0000 7f45df8a1700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d80ca370 con 0x7f45d8156ab0 2023-07-06T02:24:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dd63d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f45d8156ab0 0x7f45d8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dd63d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f45d8156ab0 0x7f45d8160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34900/0 (socket says 172.21.15.89:34900) 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dd63d700 1 -- 172.21.15.89:0/10364715 learned_addr learned my addr 172.21.15.89:0/10364715 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dce3c700 1 --1- 172.21.15.89:0/10364715 >> v1:172.21.15.89:6789/0 conn(0x7f45d8160bb0 0x7f45d80b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59802/0 (socket says 172.21.15.89:59802) 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dd63d700 1 -- 172.21.15.89:0/10364715 >> v1:172.21.15.172:6789/0 conn(0x7f45d81600b0 legacy=0x7f45d80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dd63d700 1 -- 172.21.15.89:0/10364715 >> v1:172.21.15.89:6789/0 conn(0x7f45d8160bb0 legacy=0x7f45d80b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dd63d700 1 -- 172.21.15.89:0/10364715 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80ca610 con 0x7f45d8156ab0 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45cffff700 1 -- 172.21.15.89:0/10364715 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305931694 0 0) 0x7f45d80cb3d0 con 0x7f45d8160bb0 2023-07-06T02:24:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45dd63d700 1 --2- 172.21.15.89:0/10364715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8156ab0 0x7f45d8160670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f45d40098f0 tx=0x7f45d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b89938dfc5d0ed server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45cffff700 1 -- 172.21.15.89:0/10364715 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4004230 con 0x7f45d8156ab0 2023-07-06T02:24:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45cffff700 1 -- 172.21.15.89:0/10364715 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4004390 con 0x7f45d8156ab0 2023-07-06T02:24:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.554+0000 7f45cffff700 1 -- 172.21.15.89:0/10364715 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d40036a0 con 0x7f45d8156ab0 2023-07-06T02:24:16.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.555+0000 7f45df8a1700 1 -- 172.21.15.89:0/10364715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8156ab0 msgr2=0x7f45d8160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.555+0000 7f45df8a1700 1 --2- 172.21.15.89:0/10364715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8156ab0 0x7f45d8160670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f45d40098f0 tx=0x7f45d4009c00 comp rx=0 tx=0).stop 2023-07-06T02:24:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.555+0000 7f45df8a1700 1 -- 172.21.15.89:0/10364715 shutdown_connections 2023-07-06T02:24:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.555+0000 7f45df8a1700 1 --2- 172.21.15.89:0/10364715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8156ab0 0x7f45d8160670 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.555+0000 7f45df8a1700 1 -- 172.21.15.89:0/10364715 >> 172.21.15.89:0/10364715 conn(0x7f45d80cc490 msgr2=0x7f45d80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.555+0000 7f45df8a1700 1 -- 172.21.15.89:0/10364715 shutdown_connections 2023-07-06T02:24:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.555+0000 7f45df8a1700 1 -- 172.21.15.89:0/10364715 wait complete. 2023-07-06T02:24:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.556+0000 7f45df8a1700 1 Processor -- start 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.556+0000 7f45df8a1700 1 -- start start 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45df8a1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d8156ab0 0x7f45d8149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45df8a1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d81600b0 0x7f45d80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45df8a1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45d80b5540 0x7f45d81f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45df8a1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80c79e0 con 0x7f45d81600b0 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45df8a1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d806cb00 con 0x7f45d8156ab0 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45df8a1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d806c260 con 0x7f45d80b5540 2023-07-06T02:24:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d81600b0 0x7f45d80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d81600b0 0x7f45d80b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34908/0 (socket says 172.21.15.89:34908) 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 -- 172.21.15.89:0/608602803 learned_addr learned my addr 172.21.15.89:0/608602803 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dde3e700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45d80b5540 0x7f45d81f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dd63d700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d8156ab0 0x7f45d8149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 -- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45d80b5540 msgr2=0x7f45d81f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45d80b5540 0x7f45d81f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 -- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d8156ab0 msgr2=0x7f45d8149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d8156ab0 0x7f45d8149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 -- 172.21.15.89:0/608602803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4009550 con 0x7f45d81600b0 2023-07-06T02:24:16.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.557+0000 7f45dce3c700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d81600b0 0x7f45d80b5000 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c520 tx=0x7f45c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.558+0000 7f45cdffb700 1 -- 172.21.15.89:0/608602803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8004c80 con 0x7f45d81600b0 2023-07-06T02:24:16.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.558+0000 7f45cdffb700 1 -- 172.21.15.89:0/608602803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8004de0 con 0x7f45d81600b0 2023-07-06T02:24:16.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.558+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d806c540 con 0x7f45d81600b0 2023-07-06T02:24:16.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.558+0000 7f45cdffb700 1 -- 172.21.15.89:0/608602803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80072d0 con 0x7f45d81600b0 2023-07-06T02:24:16.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.559+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d814a1d0 con 0x7f45d81600b0 2023-07-06T02:24:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.559+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8074270 con 0x7f45d81600b0 2023-07-06T02:24:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.560+0000 7f45cdffb700 1 -- 172.21.15.89:0/608602803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45c8007430 con 0x7f45d81600b0 2023-07-06T02:24:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.560+0000 7f45cdffb700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f45bc06ace0 0x7f45bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.560+0000 7f45cdffb700 1 -- 172.21.15.89:0/608602803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f45c8089230 con 0x7f45d81600b0 2023-07-06T02:24:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.560+0000 7f45dd63d700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f45bc06ace0 0x7f45bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:16.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.561+0000 7f45dd63d700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f45bc06ace0 0x7f45bc06d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009130 tx=0x7f45d4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:16.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.564+0000 7f45cdffb700 1 -- 172.21.15.89:0/608602803 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f45c8054120 con 0x7f45d81600b0 2023-07-06T02:24:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.801+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f45d80c7b20 con 0x7f45d81600b0 2023-07-06T02:24:16.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.801+0000 7f45cdffb700 1 -- 172.21.15.89:0/608602803 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f45c8057740 con 0x7f45d81600b0 2023-07-06T02:24:16.804 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T02:24:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f45bc06ace0 msgr2=0x7f45bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f45bc06ace0 0x7f45bc06d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009130 tx=0x7f45d4008030 comp rx=0 tx=0).stop 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d81600b0 msgr2=0x7f45d80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d81600b0 0x7f45d80b5000 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c520 tx=0x7f45c800c830 comp rx=0 tx=0).stop 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 shutdown_connections 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45d80b5540 0x7f45d81f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d8156ab0 0x7f45d8149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f45bc06ace0 0x7f45bc06d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 --2- 172.21.15.89:0/608602803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d81600b0 0x7f45d80b5000 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 >> 172.21.15.89:0/608602803 conn(0x7f45d80cc490 msgr2=0x7f45d80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.803+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 shutdown_connections 2023-07-06T02:24:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:16.804+0000 7f45df8a1700 1 -- 172.21.15.89:0/608602803 wait complete. 2023-07-06T02:24:16.818 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-06T02:24:16.818 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T02:24:16.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-07-06T02:24:16.819 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T02:24:16.819 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T02:24:16.819 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-06T02:24:16.820 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-06T02:24:16.820 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T02:24:16.820 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T02:24:16.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:16.821 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-07-06T02:24:16.838 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T02:24:16.838 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T02:24:16.838 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T02:24:16.838 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T02:24:16.839 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:24:16.840 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T02:24:16.840 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-07-06T02:24:16.840 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T02:24:16.840 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 41983sec preferred_lft 41983sec 2023-07-06T02:24:16.840 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5513:ae8f:6526:fe98/64 scope link noprefixroute 2023-07-06T02:24:16.840 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:24:16.840 INFO:teuthology.orchestra.run.smithi089.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 7a:a2:e5:23:3b:bc brd ff:ff:ff:ff:ff:ff 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5c4c:ff:fec3:ff8a/64 scope link 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 7a:a2:e5:23:3b:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T02:24:16.841 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::78a2:e5ff:fe23:3bbc/64 scope link 2023-07-06T02:24:16.842 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:24:16.842 INFO:teuthology.orchestra.run.smithi089.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T02:24:16.842 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether a6:6e:50:23:e7:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T02:24:16.842 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::a46e:50ff:fe23:e769/64 scope link 2023-07-06T02:24:16.842 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-07-06T02:24:16.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:16.843 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T02:24:16.902 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-07-06T02:24:16.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:16.904 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-07-06T02:24:16.961 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-07-06T02:24:16.961 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T02:24:16.962 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T02:24:16.962 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T02:24:16.962 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T02:24:16.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:16.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-07-06T02:24:16.963 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-07-06T02:24:16.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T02:24:16.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T02:24:16.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T02:24:16.965 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-07-06T02:24:17.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:17.067 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-07-06T02:24:17.083 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T02:24:17.084 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T02:24:17.084 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T02:24:17.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:17.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-07-06T02:24:17.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:24:17.150 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T02:24:17.206 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T02:24:17.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T02:24:17.206 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T02:24:17.261 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-06T02:24:17.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:17.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:24:17.346 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:24:17.346 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-06T02:24:17.347 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:24:17.355 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:24:17.355 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-06T02:24:17.355 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:24:17.356 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:24:17.356 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:24:17.380 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-06T02:24:17.380 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-06T02:24:17.381 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-07-06T02:24:17.381 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-07-06T02:24:17.381 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T02:24:17.381 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-07-06T02:24:17.381 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-07-06T02:24:17.381 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=b61ed025-dcde-49f5-a87d-8904066cfa54" will pass to kernel 2023-07-06T02:24:17.384 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T02:24:17.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:17.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T02:24:17.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:17.415 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T02:24:17.495 INFO:teuthology.orchestra.run.smithi089.stdout:client5679 2023-07-06T02:24:17.495 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:24:17.495 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:24:17.552 DEBUG:teuthology.orchestra.run.smithi089:> mktemp -d --suffix=xfstests-dev 2023-07-06T02:24:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.KkaxlHRTc5xfstests-dev 2023-07-06T02:24:17.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:17.609 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.KkaxlHRTc5xfstests-dev) 2023-07-06T02:24:17.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:17.676 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.KkaxlHRTc5xfstests-dev) 2023-07-06T02:24:17.732 INFO:teuthology.orchestra.run.smithi089.stderr:Cloning into '/tmp/tmp.KkaxlHRTc5xfstests-dev'... 2023-07-06T02:24:18.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:18.428 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-07-06T02:24:18.488 DEBUG:teuthology.orchestra.run.smithi089:> mktemp -d --suffix=test 2023-07-06T02:24:18.546 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.aPxbH4RFvvtest 2023-07-06T02:24:18.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:24:18.547 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-07-06T02:24:18.605 DEBUG:teuthology.orchestra.run.smithi089:> mktemp -d --suffix=scratch 2023-07-06T02:24:18.661 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:24:18.662 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "rhel" and "8.6". 2023-07-06T02:24:18.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc indent libtool lvm2 make psmisc quota sed xfsdump xfsprogs libacl-devel libattr-devel libaio-devel libuuid-devel xfsprogs-devel python2 sqlite xfsprogs-qa-devel 2023-07-06T02:24:19.056 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-07-06T02:24:19.889 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:04:50 ago on Thu 06 Jul 2023 02:19:29 AM UTC. 2023-07-06T02:24:21.178 INFO:teuthology.orchestra.run.smithi089.stdout:Package acl-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T02:24:21.179 INFO:teuthology.orchestra.run.smithi089.stdout:Package attr-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T02:24:21.180 INFO:teuthology.orchestra.run.smithi089.stdout:Package automake-1.16.1-7.el8.noarch is already installed. 2023-07-06T02:24:21.181 INFO:teuthology.orchestra.run.smithi089.stdout:Package bc-1.07.1-5.el8.x86_64 is already installed. 2023-07-06T02:24:21.182 INFO:teuthology.orchestra.run.smithi089.stdout:Package dbench-4.0-20.el8.x86_64 is already installed. 2023-07-06T02:24:21.184 INFO:teuthology.orchestra.run.smithi089.stdout:Package e2fsprogs-1.45.6-4.el8.x86_64 is already installed. 2023-07-06T02:24:21.187 INFO:teuthology.orchestra.run.smithi089.stdout:Package gawk-4.2.1-4.el8.x86_64 is already installed. 2023-07-06T02:24:21.188 INFO:teuthology.orchestra.run.smithi089.stdout:Package gcc-8.5.0-10.1.el8_6.x86_64 is already installed. 2023-07-06T02:24:21.190 INFO:teuthology.orchestra.run.smithi089.stdout:Package libtool-2.4.6-25.el8.x86_64 is already installed. 2023-07-06T02:24:21.191 INFO:teuthology.orchestra.run.smithi089.stdout:Package lvm2-8:2.03.14-3.el8_6.2.x86_64 is already installed. 2023-07-06T02:24:21.193 INFO:teuthology.orchestra.run.smithi089.stdout:Package make-1:4.2.1-11.el8.x86_64 is already installed. 2023-07-06T02:24:21.194 INFO:teuthology.orchestra.run.smithi089.stdout:Package psmisc-23.1-5.el8.x86_64 is already installed. 2023-07-06T02:24:21.195 INFO:teuthology.orchestra.run.smithi089.stdout:Package quota-1:4.04-14.el8.x86_64 is already installed. 2023-07-06T02:24:21.196 INFO:teuthology.orchestra.run.smithi089.stdout:Package sed-4.5-5.el8.x86_64 is already installed. 2023-07-06T02:24:21.197 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T02:24:21.198 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T02:24:21.199 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T02:24:21.201 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T02:24:21.202 INFO:teuthology.orchestra.run.smithi089.stdout:Package libaio-devel-0.3.112-1.el8.x86_64 is already installed. 2023-07-06T02:24:21.203 INFO:teuthology.orchestra.run.smithi089.stdout:Package libuuid-devel-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T02:24:21.204 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-devel-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T02:24:21.206 INFO:teuthology.orchestra.run.smithi089.stdout:Package sqlite-3.26.0-15.el8.x86_64 is already installed. 2023-07-06T02:24:21.211 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T02:24:21.325 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-07-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi089.stdout:==================================================================================================================== 2023-07-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-07-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi089.stdout:==================================================================================================================== 2023-07-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-07-06T02:24:21.328 INFO:teuthology.orchestra.run.smithi089.stdout: dump x86_64 1:0.4-0.36.b46.el8 rhel-8-for-x86_64-baseos-rpms 164 k 2023-07-06T02:24:21.329 INFO:teuthology.orchestra.run.smithi089.stdout: fio x86_64 3.19-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T02:24:21.329 INFO:teuthology.orchestra.run.smithi089.stdout: indent x86_64 2.2.13-2.el8 epel 187 k 2023-07-06T02:24:21.329 INFO:teuthology.orchestra.run.smithi089.stdout: python2 x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T02:24:21.329 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-07-06T02:24:21.329 INFO:teuthology.orchestra.run.smithi089.stdout: sqlite x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 668 k 2023-07-06T02:24:21.329 INFO:teuthology.orchestra.run.smithi089.stdout: sqlite-libs x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 581 k 2023-07-06T02:24:21.329 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-07-06T02:24:21.330 INFO:teuthology.orchestra.run.smithi089.stdout: daxctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 42 k 2023-07-06T02:24:21.330 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemblk x86_64 1.6.1-1.el8 rhel-8-for-x86_64-appstream-rpms 92 k 2023-07-06T02:24:21.330 INFO:teuthology.orchestra.run.smithi089.stdout: ndctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 79 k 2023-07-06T02:24:21.330 INFO:teuthology.orchestra.run.smithi089.stdout: python2-libs x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 6.0 M 2023-07-06T02:24:21.330 INFO:teuthology.orchestra.run.smithi089.stdout: python2-pip-wheel noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 892 k 2023-07-06T02:24:21.330 INFO:teuthology.orchestra.run.smithi089.stdout: python2-setuptools-wheel noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 287 k 2023-07-06T02:24:21.330 INFO:teuthology.orchestra.run.smithi089.stdout: python2-wheel-wheel noarch 1:0.31.1-3.module+el8.5.0+12203+77770ab7 rhel-8-for-x86_64-appstream-rpms 45 k 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout: rmt x86_64 2:1.5.3-13.el8 rhel-8-for-x86_64-baseos-rpms 51 k 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout: python2-pip noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 1.6 M 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout: python2-setuptools noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 642 k 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling module streams: 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout: python27 2.7 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:21.331 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-07-06T02:24:21.332 INFO:teuthology.orchestra.run.smithi089.stdout:==================================================================================================================== 2023-07-06T02:24:21.332 INFO:teuthology.orchestra.run.smithi089.stdout:Install 14 Packages 2023-07-06T02:24:21.332 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 2 Packages 2023-07-06T02:24:21.332 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:21.332 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 12 M 2023-07-06T02:24:21.333 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-07-06T02:24:21.468 INFO:teuthology.orchestra.run.smithi089.stdout:(1/16): rmt-1.5.3-13.el8.x86_64.rpm 558 kB/s | 51 kB 00:00 2023-07-06T02:24:21.493 INFO:teuthology.orchestra.run.smithi089.stdout:(2/16): dump-0.4-0.36.b46.el8.x86_64.rpm 1.4 MB/s | 164 kB 00:00 2023-07-06T02:24:21.518 INFO:teuthology.orchestra.run.smithi089.stdout:(3/16): daxctl-libs-71.1-3.el8.x86_64.rpm 836 kB/s | 42 kB 00:00 2023-07-06T02:24:21.552 INFO:teuthology.orchestra.run.smithi089.stdout:(4/16): ndctl-libs-71.1-3.el8.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-07-06T02:24:21.577 INFO:teuthology.orchestra.run.smithi089.stdout:(5/16): python2-setuptools-wheel-39.0.1-13.modu 4.7 MB/s | 287 kB 00:00 2023-07-06T02:24:21.611 INFO:teuthology.orchestra.run.smithi089.stdout:(6/16): python2-setuptools-39.0.1-13.module+el8 11 MB/s | 642 kB 00:00 2023-07-06T02:24:21.636 INFO:teuthology.orchestra.run.smithi089.stdout:(7/16): python2-wheel-wheel-0.31.1-3.module+el8 760 kB/s | 45 kB 00:00 2023-07-06T02:24:21.762 INFO:teuthology.orchestra.run.smithi089.stdout:(8/16): fio-3.19-3.el8.x86_64.rpm 3.5 MB/s | 538 kB 00:00 2023-07-06T02:24:21.787 INFO:teuthology.orchestra.run.smithi089.stdout:(9/16): indent-2.2.13-2.el8.x86_64.rpm 453 kB/s | 187 kB 00:00 2023-07-06T02:24:21.812 INFO:teuthology.orchestra.run.smithi089.stdout:(10/16): libpmemblk-1.6.1-1.el8.x86_64.rpm 523 kB/s | 92 kB 00:00 2023-07-06T02:24:21.846 INFO:teuthology.orchestra.run.smithi089.stdout:(11/16): python2-2.7.18-10.module+el8.6.0+14191 1.3 MB/s | 110 kB 00:00 2023-07-06T02:24:21.896 INFO:teuthology.orchestra.run.smithi089.stdout:(12/16): python2-pip-9.0.3-19.module+el8.6.0+13 21 MB/s | 1.6 MB 00:00 2023-07-06T02:24:21.939 INFO:teuthology.orchestra.run.smithi089.stdout:(13/16): python2-pip-wheel-9.0.3-19.module+el8. 9.5 MB/s | 892 kB 00:00 2023-07-06T02:24:21.972 INFO:teuthology.orchestra.run.smithi089.stdout:(14/16): sqlite-3.26.0-16.el8_6.x86_64.rpm 8.7 MB/s | 668 kB 00:00 2023-07-06T02:24:22.006 INFO:teuthology.orchestra.run.smithi089.stdout:(15/16): sqlite-libs-3.26.0-16.el8_6.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2023-07-06T02:24:22.089 INFO:teuthology.orchestra.run.smithi089.stdout:(16/16): python2-libs-2.7.18-10.module+el8.6.0+ 20 MB/s | 6.0 MB 00:00 2023-07-06T02:24:22.090 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-07-06T02:24:22.091 INFO:teuthology.orchestra.run.smithi089.stdout:Total 16 MB/s | 12 MB 00:00 2023-07-06T02:24:22.296 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-07-06T02:24:22.336 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-07-06T02:24:22.336 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-07-06T02:24:22.519 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-07-06T02:24:22.524 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-07-06T02:24:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-07-06T02:24:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : sqlite-libs-3.26.0-16.el8_6.x86_64 1/18 2023-07-06T02:24:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : daxctl-libs-71.1-3.el8.x86_64 2/18 2023-07-06T02:24:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: daxctl-libs-71.1-3.el8.x86_64 2/18 2023-07-06T02:24:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ndctl-libs-71.1-3.el8.x86_64 3/18 2023-07-06T02:24:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ndctl-libs-71.1-3.el8.x86_64 3/18 2023-07-06T02:24:24.112 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-07-06T02:24:24.209 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-07-06T02:24:24.391 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 5/18 2023-07-06T02:24:24.669 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 6/18 2023-07-06T02:24:25.277 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 7/18 2023-07-06T02:24:25.520 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 8/18 2023-07-06T02:24:25.705 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 9/18 2023-07-06T02:24:26.828 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 9/18 2023-07-06T02:24:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 10/18 2023-07-06T02:24:27.329 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rmt-2:1.5.3-13.el8.x86_64 11/18 2023-07-06T02:24:27.621 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : dump-1:0.4-0.36.b46.el8.x86_64 12/18 2023-07-06T02:24:27.875 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fio-3.19-3.el8.x86_64 13/18 2023-07-06T02:24:28.065 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : sqlite-3.26.0-16.el8_6.x86_64 14/18 2023-07-06T02:24:28.236 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 15/18 2023-07-06T02:24:28.412 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : indent-2.2.13-2.el8.x86_64 16/18 2023-07-06T02:24:28.553 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 17/18 2023-07-06T02:24:28.646 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:24:28.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:24:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-07-06T02:24:29.042 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-07-06T02:24:29.042 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : indent-2.2.13-2.el8.x86_64 1/18 2023-07-06T02:24:29.042 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : dump-1:0.4-0.36.b46.el8.x86_64 2/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rmt-2:1.5.3-13.el8.x86_64 3/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : daxctl-libs-71.1-3.el8.x86_64 4/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ndctl-libs-71.1-3.el8.x86_64 5/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 6/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 7/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 8/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fio-3.19-3.el8.x86_64 9/18 2023-07-06T02:24:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemblk-1.6.1-1.el8.x86_64 10/18 2023-07-06T02:24:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 11/18 2023-07-06T02:24:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 12/18 2023-07-06T02:24:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 13/18 2023-07-06T02:24:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 14/18 2023-07-06T02:24:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : sqlite-3.26.0-16.el8_6.x86_64 15/18 2023-07-06T02:24:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 16/18 2023-07-06T02:24:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : sqlite-libs-3.26.0-16.el8_6.x86_64 17/18 2023-07-06T02:24:30.280 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-07-06T02:24:30.280 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:05:01 ago on Thu 06 Jul 2023 02:19:29 AM UTC. 2023-07-06T02:24:33.044 INFO:teuthology.orchestra.run.smithi089.stdout:Installed products updated. 2023-07-06T02:24:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:33.238 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-07-06T02:24:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: sqlite-3.26.0-16.el8_6.x86_64 sqlite-libs-3.26.0-16.el8_6.x86_64 2023-07-06T02:24:33.239 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-07-06T02:24:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: daxctl-libs-71.1-3.el8.x86_64 2023-07-06T02:24:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: dump-1:0.4-0.36.b46.el8.x86_64 2023-07-06T02:24:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: fio-3.19-3.el8.x86_64 2023-07-06T02:24:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: indent-2.2.13-2.el8.x86_64 2023-07-06T02:24:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemblk-1.6.1-1.el8.x86_64 2023-07-06T02:24:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: ndctl-libs-71.1-3.el8.x86_64 2023-07-06T02:24:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-07-06T02:24:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-07-06T02:24:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-07-06T02:24:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-07-06T02:24:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: python2-setuptools-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-07-06T02:24:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: python2-setuptools-wheel-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-07-06T02:24:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+12203+77770ab7.noarch 2023-07-06T02:24:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: rmt-2:1.5.3-13.el8.x86_64 2023-07-06T02:24:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:33.243 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-07-06T02:24:33.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo useradd -m fsgqa 2023-07-06T02:24:33.456 INFO:teuthology.orchestra.run.smithi089.stderr:useradd: user 'fsgqa' already exists 2023-07-06T02:24:33.457 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-07-06T02:24:33.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo groupadd fsgqa 2023-07-06T02:24:33.523 INFO:teuthology.orchestra.run.smithi089.stderr:groupadd: group 'fsgqa' already exists 2023-07-06T02:24:33.524 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-07-06T02:24:33.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo useradd fsgqa2 2023-07-06T02:24:34.483 DEBUG:teuthology.orchestra.run.smithi089:> sudo useradd 123456-fsgqa 2023-07-06T02:24:35.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-07-06T02:24:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.467+0000 7f81f494a700 1 Processor -- start 2023-07-06T02:24:35.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.467+0000 7f81f494a700 1 -- start start 2023-07-06T02:24:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81f494a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f81f0070580 0x7f81f0070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81f494a700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81f00ccd40 0x7f81f00ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81f494a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f0070e90 con 0x7f81f00cc7f0 2023-07-06T02:24:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81f494a700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f006b2a0 con 0x7f81f00ccd40 2023-07-06T02:24:35.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81f494a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f006b580 con 0x7f81f0070580 2023-07-06T02:24:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81eed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81f00ccd40 0x7f81f00ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81eed9d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81f00ccd40 0x7f81f00ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34940/0 (socket says 172.21.15.89:34940) 2023-07-06T02:24:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.468+0000 7f81eed9d700 1 -- 172.21.15.89:0/275689778 learned_addr learned my addr 172.21.15.89:0/275689778 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81ee59c700 1 --2- 172.21.15.89:0/275689778 >> v2:172.21.15.172:3300/0 conn(0x7f81f0070580 0x7f81f0070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81eed9d700 1 -- 172.21.15.89:0/275689778 >> v2:172.21.15.172:3300/0 conn(0x7f81f0070580 msgr2=0x7f81f0070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81eed9d700 1 --2- 172.21.15.89:0/275689778 >> v2:172.21.15.172:3300/0 conn(0x7f81f0070580 0x7f81f0070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81eed9d700 1 -- 172.21.15.89:0/275689778 >> v1:172.21.15.172:6789/0 conn(0x7f81f00cc7f0 legacy=0x7f81f006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81eed9d700 1 -- 172.21.15.89:0/275689778 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f006b860 con 0x7f81f00ccd40 2023-07-06T02:24:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81ee59c700 1 --2- 172.21.15.89:0/275689778 >> v2:172.21.15.172:3300/0 conn(0x7f81f0070580 0x7f81f0070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T02:24:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81eed9d700 1 --2- 172.21.15.89:0/275689778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00ccd40 0x7f81f00ca2d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f81f0070380 tx=0x7f81e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2302426d46a5a1b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81ed59a700 1 -- 172.21.15.89:0/275689778 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e800e070 con 0x7f81f00ccd40 2023-07-06T02:24:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81ed59a700 1 -- 172.21.15.89:0/275689778 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8003a20 con 0x7f81f00ccd40 2023-07-06T02:24:35.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.469+0000 7f81ed59a700 1 -- 172.21.15.89:0/275689778 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8003dd0 con 0x7f81f00ccd40 2023-07-06T02:24:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 -- 172.21.15.89:0/275689778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00ccd40 msgr2=0x7f81f00ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 --2- 172.21.15.89:0/275689778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00ccd40 0x7f81f00ca2d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f81f0070380 tx=0x7f81e800b330 comp rx=0 tx=0).stop 2023-07-06T02:24:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 -- 172.21.15.89:0/275689778 shutdown_connections 2023-07-06T02:24:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 --2- 172.21.15.89:0/275689778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00ccd40 0x7f81f00ca2d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 --2- 172.21.15.89:0/275689778 >> v2:172.21.15.172:3300/0 conn(0x7f81f0070580 0x7f81f0070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 -- 172.21.15.89:0/275689778 >> 172.21.15.89:0/275689778 conn(0x7f81f00ce870 msgr2=0x7f81f00cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 -- 172.21.15.89:0/275689778 shutdown_connections 2023-07-06T02:24:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.470+0000 7f81f494a700 1 -- 172.21.15.89:0/275689778 wait complete. 2023-07-06T02:24:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.471+0000 7f81f494a700 1 Processor -- start 2023-07-06T02:24:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.471+0000 7f81f494a700 1 -- start start 2023-07-06T02:24:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81f494a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0070580 0x7f81f014dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81f494a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00cc7f0 0x7f81f01532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81f494a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f00ccd40 0x7f81f015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81f494a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f01537e0 con 0x7f81f00cc7f0 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81f494a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f015c6b0 con 0x7f81f0070580 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81f494a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f015c0f0 con 0x7f81f00ccd40 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00cc7f0 0x7f81f01532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00cc7f0 0x7f81f01532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34944/0 (socket says 172.21.15.89:34944) 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 -- 172.21.15.89:0/2094900294 learned_addr learned my addr 172.21.15.89:0/2094900294 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81ee59c700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0070580 0x7f81f014dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81eed9d700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f00ccd40 0x7f81f015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 -- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f00ccd40 msgr2=0x7f81f015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f00ccd40 0x7f81f015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 -- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0070580 msgr2=0x7f81f014dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0070580 0x7f81f014dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 -- 172.21.15.89:0/2094900294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009b30 con 0x7f81f00cc7f0 2023-07-06T02:24:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81ee59c700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0070580 0x7f81f014dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:35.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81eed9d700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f00ccd40 0x7f81f015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.472+0000 7f81edd9b700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00cc7f0 0x7f81f01532a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f81e000d9c0 tx=0x7f81e000dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.473+0000 7f81e6ffd700 1 -- 172.21.15.89:0/2094900294 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e000ebf0 con 0x7f81f00cc7f0 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.473+0000 7f81e6ffd700 1 -- 172.21.15.89:0/2094900294 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e000ed50 con 0x7f81f00cc7f0 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.473+0000 7f81e6ffd700 1 -- 172.21.15.89:0/2094900294 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0004260 con 0x7f81f00cc7f0 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.473+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f015c3d0 con 0x7f81f00cc7f0 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.473+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f00b1a90 con 0x7f81f00cc7f0 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.474+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f0074270 con 0x7f81f00cc7f0 2023-07-06T02:24:35.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.474+0000 7f81e6ffd700 1 -- 172.21.15.89:0/2094900294 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81e00043c0 con 0x7f81f00cc7f0 2023-07-06T02:24:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.474+0000 7f81e6ffd700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f81d006ace0 0x7f81d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.475+0000 7f81e6ffd700 1 -- 172.21.15.89:0/2094900294 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f81e008b320 con 0x7f81f00cc7f0 2023-07-06T02:24:35.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.475+0000 7f81ee59c700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f81d006ace0 0x7f81d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.475+0000 7f81ee59c700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f81d006ace0 0x7f81d006d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f81d8006fd0 tx=0x7f81d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:35.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.479+0000 7f81e6ffd700 1 -- 172.21.15.89:0/2094900294 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f81e0056290 con 0x7f81f00cc7f0 2023-07-06T02:24:35.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.758+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f81f00b1c50 con 0x7f81f00cc7f0 2023-07-06T02:24:35.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.759+0000 7f81e6ffd700 1 -- 172.21.15.89:0/2094900294 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f81e005f090 con 0x7f81f00cc7f0 2023-07-06T02:24:35.761 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:24:35.761 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":1,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","modified":"2023-07-06T02:21:59.936694Z","created":"2023-07-06T02:21:59.936694Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3301","nonce":0},{"type":"v1","addr":"172.21.15.172:6790","nonce":0}]},"addr":"172.21.15.172:6790/0","public_addr":"172.21.15.172:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-07-06T02:24:35.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f81d006ace0 msgr2=0x7f81d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f81d006ace0 0x7f81d006d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f81d8006fd0 tx=0x7f81d8008040 comp rx=0 tx=0).stop 2023-07-06T02:24:35.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00cc7f0 msgr2=0x7f81f01532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00cc7f0 0x7f81f01532a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f81e000d9c0 tx=0x7f81e000dd80 comp rx=0 tx=0).stop 2023-07-06T02:24:35.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 shutdown_connections 2023-07-06T02:24:35.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f00ccd40 0x7f81f015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0070580 0x7f81f014dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f81d006ace0 0x7f81d006d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 --2- 172.21.15.89:0/2094900294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f00cc7f0 0x7f81f01532a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 >> 172.21.15.89:0/2094900294 conn(0x7f81f00ce870 msgr2=0x7f81f015cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:35.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 shutdown_connections 2023-07-06T02:24:35.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.761+0000 7f81f494a700 1 -- 172.21.15.89:0/2094900294 wait complete. 2023-07-06T02:24:35.765 INFO:teuthology.orchestra.run.smithi089.stderr:dumped monmap epoch 1 2023-07-06T02:24:35.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2023-07-06T02:24:35.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.922+0000 7f4e4649d700 1 Processor -- start 2023-07-06T02:24:35.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.922+0000 7f4e4649d700 1 -- start start 2023-07-06T02:24:35.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e4649d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e4006bdb0 0x7f4e4006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e4649d700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e4006b610 0x7f4e4006c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e4649d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e400cd240 con 0x7f4e400ceda0 2023-07-06T02:24:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e4649d700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e400cf280 con 0x7f4e4006b610 2023-07-06T02:24:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e4649d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e400cd490 con 0x7f4e4006bdb0 2023-07-06T02:24:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e4006b610 0x7f4e4006c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e4006b610 0x7f4e4006c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34950/0 (socket says 172.21.15.89:34950) 2023-07-06T02:24:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 -- 172.21.15.89:0/3780405930 learned_addr learned my addr 172.21.15.89:0/3780405930 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3ffff700 1 --2- 172.21.15.89:0/3780405930 >> v2:172.21.15.172:3300/0 conn(0x7f4e4006bdb0 0x7f4e4006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 -- 172.21.15.89:0/3780405930 >> v2:172.21.15.172:3300/0 conn(0x7f4e4006bdb0 msgr2=0x7f4e4006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 --2- 172.21.15.89:0/3780405930 >> v2:172.21.15.172:3300/0 conn(0x7f4e4006bdb0 0x7f4e4006aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 -- 172.21.15.89:0/3780405930 >> v1:172.21.15.172:6789/0 conn(0x7f4e400ceda0 legacy=0x7f4e400d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 -- 172.21.15.89:0/3780405930 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e400cd770 con 0x7f4e4006b610 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3f7fe700 1 --2- 172.21.15.89:0/3780405930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006b610 0x7f4e4006c180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4e400d3cd0 tx=0x7f4e34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ae68b9dc02ce26 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.923+0000 7f4e3effd700 1 -- 172.21.15.89:0/3780405930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34011830 con 0x7f4e4006b610 2023-07-06T02:24:35.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e3effd700 1 -- 172.21.15.89:0/3780405930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e34011990 con 0x7f4e4006b610 2023-07-06T02:24:35.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e3effd700 1 -- 172.21.15.89:0/3780405930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34011d20 con 0x7f4e4006b610 2023-07-06T02:24:35.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 -- 172.21.15.89:0/3780405930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006b610 msgr2=0x7f4e4006c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 --2- 172.21.15.89:0/3780405930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006b610 0x7f4e4006c180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4e400d3cd0 tx=0x7f4e34009bd0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 -- 172.21.15.89:0/3780405930 shutdown_connections 2023-07-06T02:24:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 --2- 172.21.15.89:0/3780405930 >> v2:172.21.15.172:3300/0 conn(0x7f4e4006bdb0 0x7f4e4006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 --2- 172.21.15.89:0/3780405930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006b610 0x7f4e4006c180 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 -- 172.21.15.89:0/3780405930 >> 172.21.15.89:0/3780405930 conn(0x7f4e400b5390 msgr2=0x7f4e400d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 -- 172.21.15.89:0/3780405930 shutdown_connections 2023-07-06T02:24:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.924+0000 7f4e4649d700 1 -- 172.21.15.89:0/3780405930 wait complete. 2023-07-06T02:24:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.925+0000 7f4e4649d700 1 Processor -- start 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e4649d700 1 -- start start 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e4649d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006bdb0 0x7f4e400cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e4649d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e400ceda0 0x7f4e401579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e4649d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e40152e60 0x7f4e4015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e4649d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400cbc50 con 0x7f4e4006bdb0 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e4649d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400cbed0 con 0x7f4e400ceda0 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e4649d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400cc420 con 0x7f4e40152e60 2023-07-06T02:24:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e3ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006bdb0 0x7f4e400cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e3ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006bdb0 0x7f4e400cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34956/0 (socket says 172.21.15.89:34956) 2023-07-06T02:24:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.926+0000 7f4e3ffff700 1 -- 172.21.15.89:0/1662047079 learned_addr learned my addr 172.21.15.89:0/1662047079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:24:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3f7fe700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e400ceda0 0x7f4e401579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e44a3a700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e40152e60 0x7f4e4015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3ffff700 1 -- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e40152e60 msgr2=0x7f4e4015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3ffff700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e40152e60 0x7f4e4015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3ffff700 1 -- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e400ceda0 msgr2=0x7f4e401579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3ffff700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e400ceda0 0x7f4e401579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3ffff700 1 -- 172.21.15.89:0/1662047079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e34009550 con 0x7f4e4006bdb0 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3f7fe700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e400ceda0 0x7f4e401579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3ffff700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006bdb0 0x7f4e400cb710 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000e9b0 tx=0x7f4e3000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3cff9700 1 -- 172.21.15.89:0/1662047079 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30009710 con 0x7f4e4006bdb0 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3cff9700 1 -- 172.21.15.89:0/1662047079 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3000c850 con 0x7f4e4006bdb0 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e3cff9700 1 -- 172.21.15.89:0/1662047079 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000cbc0 con 0x7f4e4006bdb0 2023-07-06T02:24:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e400cc700 con 0x7f4e4006bdb0 2023-07-06T02:24:35.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.927+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e400c6720 con 0x7f4e4006bdb0 2023-07-06T02:24:35.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.928+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e40074270 con 0x7f4e4006bdb0 2023-07-06T02:24:35.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.928+0000 7f4e3cff9700 1 -- 172.21.15.89:0/1662047079 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e3000cd20 con 0x7f4e4006bdb0 2023-07-06T02:24:35.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.929+0000 7f4e3cff9700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e2c06ace0 0x7f4e2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:24:35.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.929+0000 7f4e3cff9700 1 -- 172.21.15.89:0/1662047079 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4e300896a0 con 0x7f4e4006bdb0 2023-07-06T02:24:35.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.929+0000 7f4e3f7fe700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e2c06ace0 0x7f4e2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:24:35.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.929+0000 7f4e3f7fe700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e2c06ace0 0x7f4e2c06d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e400b46b0 tx=0x7f4e34008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:24:35.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:35.933+0000 7f4e3cff9700 1 -- 172.21.15.89:0/1662047079 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4e30058130 con 0x7f4e4006bdb0 2023-07-06T02:24:36.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.219+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7f4e400ce180 con 0x7f4e4006bdb0 2023-07-06T02:24:36.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.219+0000 7f4e3cff9700 1 -- 172.21.15.89:0/1662047079 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v13) v1 ==== 92+0+63 (secure 0 0 0) 0x7f4e30057cc0 con 0x7f4e4006bdb0 2023-07-06T02:24:36.221 INFO:teuthology.orchestra.run.smithi089.stdout:[client.admin] 2023-07-06T02:24:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== 2023-07-06T02:24:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e2c06ace0 msgr2=0x7f4e2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e2c06ace0 0x7f4e2c06d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e400b46b0 tx=0x7f4e34008030 comp rx=0 tx=0).stop 2023-07-06T02:24:36.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006bdb0 msgr2=0x7f4e400cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:24:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006bdb0 0x7f4e400cb710 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000e9b0 tx=0x7f4e3000ed70 comp rx=0 tx=0).stop 2023-07-06T02:24:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 shutdown_connections 2023-07-06T02:24:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e40152e60 0x7f4e4015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f4e2c06ace0 0x7f4e2c06d190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e4006bdb0 0x7f4e400cb710 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:36.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 --2- 172.21.15.89:0/1662047079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e400ceda0 0x7f4e401579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:24:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 >> 172.21.15.89:0/1662047079 conn(0x7f4e400b5390 msgr2=0x7f4e400cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:24:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 shutdown_connections 2023-07-06T02:24:36.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:24:36.221+0000 7f4e4649d700 1 -- 172.21.15.89:0/1662047079 wait complete. 2023-07-06T02:24:36.235 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:24:36.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/tmp/tmp.KkaxlHRTc5xfstests-dev/local.config 2023-07-06T02:24:36.273 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.89:6789:/test export TEST_DIR=/tmp/tmp.aPxbH4RFvvtest export SCRATCH_DEV=172.21.15.89:6789:/scratch export SCRATCH_MNT=/tmp/tmp.vhpd3PlzjEscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg==" 2023-07-06T02:24:36.273 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:24:36.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/tmp/tmp.KkaxlHRTc5xfstests-dev/ceph.exclude 2023-07-06T02:24:36.339 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo make) 2023-07-06T02:24:36.420 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize -cfi 2023-07-06T02:24:36.721 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: putting auxiliary files in '.'. 2023-07-06T02:24:36.722 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file './config.guess' 2023-07-06T02:24:36.725 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file './config.sub' 2023-07-06T02:24:36.728 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file './install-sh' 2023-07-06T02:24:36.732 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file './ltmain.sh' 2023-07-06T02:24:36.752 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-07-06T02:24:36.752 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-07-06T02:24:36.756 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-07-06T02:24:36.761 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-07-06T02:24:36.765 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-07-06T02:24:36.769 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-07-06T02:24:36.775 INFO:teuthology.orchestra.run.smithi089.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-07-06T02:24:36.776 INFO:teuthology.orchestra.run.smithi089.stdout:cp include/install-sh . 2023-07-06T02:24:36.777 INFO:teuthology.orchestra.run.smithi089.stdout:aclocal -I m4 2023-07-06T02:24:37.921 INFO:teuthology.orchestra.run.smithi089.stdout:autoheader 2023-07-06T02:24:38.227 INFO:teuthology.orchestra.run.smithi089.stdout:autoconf 2023-07-06T02:24:38.385 INFO:teuthology.orchestra.run.smithi089.stdout:./configure \ 2023-07-06T02:24:38.385 INFO:teuthology.orchestra.run.smithi089.stdout: --libexecdir=/usr/lib \ 2023-07-06T02:24:38.385 INFO:teuthology.orchestra.run.smithi089.stdout: --exec_prefix=/var/lib 2023-07-06T02:24:38.848 INFO:teuthology.orchestra.run.smithi089.stdout:checking build system type... x86_64-pc-linux-gnu 2023-07-06T02:24:38.849 INFO:teuthology.orchestra.run.smithi089.stdout:checking host system type... x86_64-pc-linux-gnu 2023-07-06T02:24:38.850 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to print strings... printf 2023-07-06T02:24:38.850 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gcc... gcc 2023-07-06T02:24:38.957 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether the C compiler works... yes 2023-07-06T02:24:38.958 INFO:teuthology.orchestra.run.smithi089.stdout:checking for C compiler default output file name... a.out 2023-07-06T02:24:38.984 INFO:teuthology.orchestra.run.smithi089.stdout:checking for suffix of executables... 2023-07-06T02:24:39.023 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether we are cross compiling... no 2023-07-06T02:24:39.040 INFO:teuthology.orchestra.run.smithi089.stdout:checking for suffix of object files... o 2023-07-06T02:24:39.056 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether we are using the GNU C compiler... yes 2023-07-06T02:24:39.071 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether gcc accepts -g... yes 2023-07-06T02:24:39.098 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gcc option to accept ISO C89... none needed 2023-07-06T02:24:39.100 INFO:teuthology.orchestra.run.smithi089.stdout:checking for a sed that does not truncate output... /bin/sed 2023-07-06T02:24:39.102 INFO:teuthology.orchestra.run.smithi089.stdout:checking for grep that handles long lines and -e... /bin/grep 2023-07-06T02:24:39.103 INFO:teuthology.orchestra.run.smithi089.stdout:checking for egrep... /bin/grep -E 2023-07-06T02:24:39.104 INFO:teuthology.orchestra.run.smithi089.stdout:checking for fgrep... /bin/grep -F 2023-07-06T02:24:39.107 INFO:teuthology.orchestra.run.smithi089.stdout:checking for ld used by gcc... /bin/ld 2023-07-06T02:24:39.109 INFO:teuthology.orchestra.run.smithi089.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2023-07-06T02:24:39.120 INFO:teuthology.orchestra.run.smithi089.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2023-07-06T02:24:39.135 INFO:teuthology.orchestra.run.smithi089.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2023-07-06T02:24:39.136 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether ln -s works... yes 2023-07-06T02:24:39.142 INFO:teuthology.orchestra.run.smithi089.stdout:checking the maximum length of command line arguments... 1572864 2023-07-06T02:24:39.144 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-07-06T02:24:39.144 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-07-06T02:24:39.145 INFO:teuthology.orchestra.run.smithi089.stdout:checking for /bin/ld option to reload object files... -r 2023-07-06T02:24:39.145 INFO:teuthology.orchestra.run.smithi089.stdout:checking for objdump... objdump 2023-07-06T02:24:39.145 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to recognize dependent libraries... pass_all 2023-07-06T02:24:39.146 INFO:teuthology.orchestra.run.smithi089.stdout:checking for dlltool... no 2023-07-06T02:24:39.146 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-07-06T02:24:39.146 INFO:teuthology.orchestra.run.smithi089.stdout:checking for ar... ar 2023-07-06T02:24:39.167 INFO:teuthology.orchestra.run.smithi089.stdout:checking for archiver @FILE support... @ 2023-07-06T02:24:39.167 INFO:teuthology.orchestra.run.smithi089.stdout:checking for strip... strip 2023-07-06T02:24:39.168 INFO:teuthology.orchestra.run.smithi089.stdout:checking for ranlib... ranlib 2023-07-06T02:24:39.169 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gawk... gawk 2023-07-06T02:24:39.217 INFO:teuthology.orchestra.run.smithi089.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2023-07-06T02:24:39.219 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sysroot... no 2023-07-06T02:24:39.221 INFO:teuthology.orchestra.run.smithi089.stdout:checking for a working dd... /bin/dd 2023-07-06T02:24:39.224 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2023-07-06T02:24:39.236 INFO:teuthology.orchestra.run.smithi089.stdout:checking for mt... no 2023-07-06T02:24:39.238 INFO:teuthology.orchestra.run.smithi089.stdout:checking if : is a manifest tool... no 2023-07-06T02:24:39.261 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to run the C preprocessor... gcc -E 2023-07-06T02:24:39.377 INFO:teuthology.orchestra.run.smithi089.stdout:checking for ANSI C header files... yes 2023-07-06T02:24:39.396 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/types.h... yes 2023-07-06T02:24:39.419 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/stat.h... yes 2023-07-06T02:24:39.442 INFO:teuthology.orchestra.run.smithi089.stdout:checking for stdlib.h... yes 2023-07-06T02:24:39.465 INFO:teuthology.orchestra.run.smithi089.stdout:checking for string.h... yes 2023-07-06T02:24:39.489 INFO:teuthology.orchestra.run.smithi089.stdout:checking for memory.h... yes 2023-07-06T02:24:39.512 INFO:teuthology.orchestra.run.smithi089.stdout:checking for strings.h... yes 2023-07-06T02:24:39.539 INFO:teuthology.orchestra.run.smithi089.stdout:checking for inttypes.h... yes 2023-07-06T02:24:39.563 INFO:teuthology.orchestra.run.smithi089.stdout:checking for stdint.h... yes 2023-07-06T02:24:39.590 INFO:teuthology.orchestra.run.smithi089.stdout:checking for unistd.h... yes 2023-07-06T02:24:39.615 INFO:teuthology.orchestra.run.smithi089.stdout:checking for dlfcn.h... yes 2023-07-06T02:24:39.618 INFO:teuthology.orchestra.run.smithi089.stdout:checking for objdir... .libs 2023-07-06T02:24:39.682 INFO:teuthology.orchestra.run.smithi089.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-07-06T02:24:39.683 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-07-06T02:24:39.697 INFO:teuthology.orchestra.run.smithi089.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-07-06T02:24:39.715 INFO:teuthology.orchestra.run.smithi089.stdout:checking if gcc static flag -static works... no 2023-07-06T02:24:39.735 INFO:teuthology.orchestra.run.smithi089.stdout:checking if gcc supports -c -o file.o... yes 2023-07-06T02:24:39.735 INFO:teuthology.orchestra.run.smithi089.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-07-06T02:24:39.741 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-07-06T02:24:39.765 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether -lc should be explicitly linked in... no 2023-07-06T02:24:39.808 INFO:teuthology.orchestra.run.smithi089.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-07-06T02:24:39.808 INFO:teuthology.orchestra.run.smithi089.stdout:checking how to hardcode library paths into programs... immediate 2023-07-06T02:24:39.810 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether stripping libraries is possible... yes 2023-07-06T02:24:39.810 INFO:teuthology.orchestra.run.smithi089.stdout:checking if libtool supports shared libraries... yes 2023-07-06T02:24:39.810 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether to build shared libraries... yes 2023-07-06T02:24:39.810 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether to build static libraries... yes 2023-07-06T02:24:39.814 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gcc... (cached) gcc 2023-07-06T02:24:39.829 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-07-06T02:24:39.829 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether gcc accepts -g... (cached) yes 2023-07-06T02:24:39.830 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-07-06T02:24:39.830 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gmake... /usr/bin/gmake 2023-07-06T02:24:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:./configure: line 12070: test: $(SHELL): binary operator expected 2023-07-06T02:24:39.831 INFO:teuthology.orchestra.run.smithi089.stdout:checking for tar... /bin/tar 2023-07-06T02:24:39.831 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gzip... /bin/gzip 2023-07-06T02:24:39.837 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether gcc -MM is supported... yes 2023-07-06T02:24:39.838 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sort... /bin/sort 2023-07-06T02:24:39.838 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether ln -s works... yes 2023-07-06T02:24:39.838 INFO:teuthology.orchestra.run.smithi089.stdout:checking for rpm... /bin/rpm 2023-07-06T02:24:39.841 INFO:teuthology.orchestra.run.smithi089.stdout:checking for rpmbuild... no 2023-07-06T02:24:39.873 INFO:teuthology.orchestra.run.smithi089.stdout:checking assert.h usability... yes 2023-07-06T02:24:39.879 INFO:teuthology.orchestra.run.smithi089.stdout:checking assert.h presence... yes 2023-07-06T02:24:39.880 INFO:teuthology.orchestra.run.smithi089.stdout:checking for assert.h... yes 2023-07-06T02:24:39.905 INFO:teuthology.orchestra.run.smithi089.stdout:checking bstring.h usability... no 2023-07-06T02:24:39.914 INFO:teuthology.orchestra.run.smithi089.stdout:checking bstring.h presence... no 2023-07-06T02:24:39.914 INFO:teuthology.orchestra.run.smithi089.stdout:checking for bstring.h... no 2023-07-06T02:24:39.938 INFO:teuthology.orchestra.run.smithi089.stdout:checking libgen.h usability... yes 2023-07-06T02:24:39.945 INFO:teuthology.orchestra.run.smithi089.stdout:checking libgen.h presence... yes 2023-07-06T02:24:39.945 INFO:teuthology.orchestra.run.smithi089.stdout:checking for libgen.h... yes 2023-07-06T02:24:39.972 INFO:teuthology.orchestra.run.smithi089.stdout:checking dirent.h usability... yes 2023-07-06T02:24:39.979 INFO:teuthology.orchestra.run.smithi089.stdout:checking dirent.h presence... yes 2023-07-06T02:24:39.979 INFO:teuthology.orchestra.run.smithi089.stdout:checking for dirent.h... yes 2023-07-06T02:24:40.030 INFO:teuthology.orchestra.run.smithi089.stdout:checking errno.h usability... yes 2023-07-06T02:24:40.036 INFO:teuthology.orchestra.run.smithi089.stdout:checking errno.h presence... yes 2023-07-06T02:24:40.037 INFO:teuthology.orchestra.run.smithi089.stdout:checking for errno.h... yes 2023-07-06T02:24:40.063 INFO:teuthology.orchestra.run.smithi089.stdout:checking malloc.h usability... yes 2023-07-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi089.stdout:checking malloc.h presence... yes 2023-07-06T02:24:40.071 INFO:teuthology.orchestra.run.smithi089.stdout:checking for malloc.h... yes 2023-07-06T02:24:40.096 INFO:teuthology.orchestra.run.smithi089.stdout:checking uuid.h usability... no 2023-07-06T02:24:40.105 INFO:teuthology.orchestra.run.smithi089.stdout:checking uuid.h presence... no 2023-07-06T02:24:40.105 INFO:teuthology.orchestra.run.smithi089.stdout:checking for uuid.h... no 2023-07-06T02:24:40.131 INFO:teuthology.orchestra.run.smithi089.stdout:checking uuid/uuid.h usability... yes 2023-07-06T02:24:40.139 INFO:teuthology.orchestra.run.smithi089.stdout:checking uuid/uuid.h presence... yes 2023-07-06T02:24:40.139 INFO:teuthology.orchestra.run.smithi089.stdout:checking for uuid/uuid.h... yes 2023-07-06T02:24:40.164 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/uuid.h usability... no 2023-07-06T02:24:40.173 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/uuid.h presence... no 2023-07-06T02:24:40.174 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/uuid.h... no 2023-07-06T02:24:40.198 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/file.h usability... yes 2023-07-06T02:24:40.205 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/file.h presence... yes 2023-07-06T02:24:40.206 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/file.h... yes 2023-07-06T02:24:40.232 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/fcntl.h usability... yes 2023-07-06T02:24:40.239 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/fcntl.h presence... yes 2023-07-06T02:24:40.239 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/fcntl.h... yes 2023-07-06T02:24:40.265 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/syssgi.h usability... no 2023-07-06T02:24:40.274 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/syssgi.h presence... no 2023-07-06T02:24:40.274 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/syssgi.h... no 2023-07-06T02:24:40.316 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/param.h usability... yes 2023-07-06T02:24:40.325 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/param.h presence... yes 2023-07-06T02:24:40.325 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/param.h... yes 2023-07-06T02:24:40.329 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/stat.h... (cached) yes 2023-07-06T02:24:40.357 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/statvfs.h usability... yes 2023-07-06T02:24:40.364 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/statvfs.h presence... yes 2023-07-06T02:24:40.364 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/statvfs.h... yes 2023-07-06T02:24:40.389 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/time.h usability... yes 2023-07-06T02:24:40.397 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/time.h presence... yes 2023-07-06T02:24:40.397 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/time.h... yes 2023-07-06T02:24:40.439 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/ioctl.h usability... yes 2023-07-06T02:24:40.446 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/ioctl.h presence... yes 2023-07-06T02:24:40.447 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/ioctl.h... yes 2023-07-06T02:24:40.475 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/wait.h usability... yes 2023-07-06T02:24:40.483 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/wait.h presence... yes 2023-07-06T02:24:40.484 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/wait.h... yes 2023-07-06T02:24:40.488 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/types.h... (cached) yes 2023-07-06T02:24:40.493 INFO:teuthology.orchestra.run.smithi089.stdout:checking for strings.h... (cached) yes 2023-07-06T02:24:40.524 INFO:teuthology.orchestra.run.smithi089.stdout:checking err.h usability... yes 2023-07-06T02:24:40.531 INFO:teuthology.orchestra.run.smithi089.stdout:checking err.h presence... yes 2023-07-06T02:24:40.531 INFO:teuthology.orchestra.run.smithi089.stdout:checking for err.h... yes 2023-07-06T02:24:40.557 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/falloc.h usability... yes 2023-07-06T02:24:40.563 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/falloc.h presence... yes 2023-07-06T02:24:40.563 INFO:teuthology.orchestra.run.smithi089.stdout:checking for linux/falloc.h... yes 2023-07-06T02:24:40.591 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-07-06T02:24:40.600 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-07-06T02:24:40.600 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/fs/xfs_fsops.h... no 2023-07-06T02:24:40.622 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/fs/xfs_itable.h usability... no 2023-07-06T02:24:40.631 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/fs/xfs_itable.h presence... no 2023-07-06T02:24:40.632 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/fs/xfs_itable.h... no 2023-07-06T02:24:40.654 INFO:teuthology.orchestra.run.smithi089.stdout:checking xfs/platform_defs.h usability... no 2023-07-06T02:24:40.664 INFO:teuthology.orchestra.run.smithi089.stdout:checking xfs/platform_defs.h presence... no 2023-07-06T02:24:40.664 INFO:teuthology.orchestra.run.smithi089.stdout:checking for xfs/platform_defs.h... no 2023-07-06T02:24:40.686 INFO:teuthology.orchestra.run.smithi089.stdout:checking btrfs/ioctl.h usability... no 2023-07-06T02:24:40.695 INFO:teuthology.orchestra.run.smithi089.stdout:checking btrfs/ioctl.h presence... no 2023-07-06T02:24:40.695 INFO:teuthology.orchestra.run.smithi089.stdout:checking for btrfs/ioctl.h... no 2023-07-06T02:24:40.718 INFO:teuthology.orchestra.run.smithi089.stdout:checking cifs/ioctl.h usability... no 2023-07-06T02:24:40.727 INFO:teuthology.orchestra.run.smithi089.stdout:checking cifs/ioctl.h presence... no 2023-07-06T02:24:40.727 INFO:teuthology.orchestra.run.smithi089.stdout:checking for cifs/ioctl.h... no 2023-07-06T02:24:40.752 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/mman.h usability... yes 2023-07-06T02:24:40.760 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/mman.h presence... yes 2023-07-06T02:24:40.760 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/mman.h... yes 2023-07-06T02:24:40.776 INFO:teuthology.orchestra.run.smithi089.stdout:checking for xfs/xfs_log_format.h... no 2023-07-06T02:24:40.819 INFO:teuthology.orchestra.run.smithi089.stdout:checking for uuid_compare... no 2023-07-06T02:24:40.848 INFO:teuthology.orchestra.run.smithi089.stdout:checking for uuid_compare in -luuid... yes 2023-07-06T02:24:40.903 INFO:teuthology.orchestra.run.smithi089.stdout:checking for xfs/xfs.h... yes 2023-07-06T02:24:40.916 INFO:teuthology.orchestra.run.smithi089.stdout:checking for xfs/libxfs.h... no 2023-07-06T02:24:40.931 INFO:teuthology.orchestra.run.smithi089.stdout:checking whether xlog_assign_lsn is declared... no 2023-07-06T02:24:40.957 INFO:teuthology.orchestra.run.smithi089.stdout:checking for xfs/xqm.h... yes 2023-07-06T02:24:41.001 INFO:teuthology.orchestra.run.smithi089.stdout:checking xfsctl from xfs/xfs.h... ok 2023-07-06T02:24:41.023 INFO:teuthology.orchestra.run.smithi089.stdout:checking xfs/handle.h usability... yes 2023-07-06T02:24:41.029 INFO:teuthology.orchestra.run.smithi089.stdout:checking xfs/handle.h presence... yes 2023-07-06T02:24:41.029 INFO:teuthology.orchestra.run.smithi089.stdout:checking for xfs/handle.h... yes 2023-07-06T02:24:41.063 INFO:teuthology.orchestra.run.smithi089.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-07-06T02:24:41.097 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/acl.h usability... yes 2023-07-06T02:24:41.105 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/acl.h presence... yes 2023-07-06T02:24:41.105 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/acl.h... yes 2023-07-06T02:24:41.139 INFO:teuthology.orchestra.run.smithi089.stdout:checking acl/libacl.h usability... yes 2023-07-06T02:24:41.147 INFO:teuthology.orchestra.run.smithi089.stdout:checking acl/libacl.h presence... yes 2023-07-06T02:24:41.148 INFO:teuthology.orchestra.run.smithi089.stdout:checking for acl/libacl.h... yes 2023-07-06T02:24:41.177 INFO:teuthology.orchestra.run.smithi089.stdout:checking for acl_init in -lacl... yes 2023-07-06T02:24:41.199 INFO:teuthology.orchestra.run.smithi089.stdout:checking gdbm-ndbm.h usability... no 2023-07-06T02:24:41.209 INFO:teuthology.orchestra.run.smithi089.stdout:checking gdbm-ndbm.h presence... no 2023-07-06T02:24:41.209 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gdbm-ndbm.h... no 2023-07-06T02:24:41.230 INFO:teuthology.orchestra.run.smithi089.stdout:checking gdbm/ndbm.h usability... no 2023-07-06T02:24:41.240 INFO:teuthology.orchestra.run.smithi089.stdout:checking gdbm/ndbm.h presence... no 2023-07-06T02:24:41.240 INFO:teuthology.orchestra.run.smithi089.stdout:checking for gdbm/ndbm.h... no 2023-07-06T02:24:41.261 INFO:teuthology.orchestra.run.smithi089.stdout:checking ndbm.h usability... no 2023-07-06T02:24:41.270 INFO:teuthology.orchestra.run.smithi089.stdout:checking ndbm.h presence... no 2023-07-06T02:24:41.270 INFO:teuthology.orchestra.run.smithi089.stdout:checking for ndbm.h... no 2023-07-06T02:24:41.294 INFO:teuthology.orchestra.run.smithi089.stdout:checking libaio.h usability... yes 2023-07-06T02:24:41.304 INFO:teuthology.orchestra.run.smithi089.stdout:checking libaio.h presence... yes 2023-07-06T02:24:41.304 INFO:teuthology.orchestra.run.smithi089.stdout:checking for libaio.h... yes 2023-07-06T02:24:41.326 INFO:teuthology.orchestra.run.smithi089.stdout:checking liburing.h usability... no 2023-07-06T02:24:41.335 INFO:teuthology.orchestra.run.smithi089.stdout:checking liburing.h presence... no 2023-07-06T02:24:41.335 INFO:teuthology.orchestra.run.smithi089.stdout:checking for liburing.h... no 2023-07-06T02:24:41.356 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/capability.h usability... no 2023-07-06T02:24:41.365 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/capability.h presence... no 2023-07-06T02:24:41.365 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/capability.h... no 2023-07-06T02:24:41.387 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/fiemap.h usability... yes 2023-07-06T02:24:41.394 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/fiemap.h presence... yes 2023-07-06T02:24:41.394 INFO:teuthology.orchestra.run.smithi089.stdout:checking for linux/fiemap.h... yes 2023-07-06T02:24:41.425 INFO:teuthology.orchestra.run.smithi089.stdout:checking for fallocate... yes 2023-07-06T02:24:41.456 INFO:teuthology.orchestra.run.smithi089.stdout:checking for open_by_handle_at... yes 2023-07-06T02:24:41.480 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/prctl.h usability... yes 2023-07-06T02:24:41.487 INFO:teuthology.orchestra.run.smithi089.stdout:checking sys/prctl.h presence... yes 2023-07-06T02:24:41.487 INFO:teuthology.orchestra.run.smithi089.stdout:checking for sys/prctl.h... yes 2023-07-06T02:24:41.510 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/fs.h usability... yes 2023-07-06T02:24:41.517 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/fs.h presence... yes 2023-07-06T02:24:41.518 INFO:teuthology.orchestra.run.smithi089.stdout:checking for linux/fs.h... yes 2023-07-06T02:24:41.539 INFO:teuthology.orchestra.run.smithi089.stdout:checking btrfsutil.h usability... no 2023-07-06T02:24:41.549 INFO:teuthology.orchestra.run.smithi089.stdout:checking btrfsutil.h presence... no 2023-07-06T02:24:41.549 INFO:teuthology.orchestra.run.smithi089.stdout:checking for btrfsutil.h... no 2023-07-06T02:24:41.582 INFO:teuthology.orchestra.run.smithi089.stdout:checking for copy_file_range... yes 2023-07-06T02:24:41.616 INFO:teuthology.orchestra.run.smithi089.stdout:checking for SEEK_DATA... yes 2023-07-06T02:24:41.659 INFO:teuthology.orchestra.run.smithi089.stdout:checking for BMV_OF_SHARED... yes 2023-07-06T02:24:41.693 INFO:teuthology.orchestra.run.smithi089.stdout:checking for nftw... yes 2023-07-06T02:24:41.726 INFO:teuthology.orchestra.run.smithi089.stdout:checking for RLIMIT_NOFILE... yes 2023-07-06T02:24:41.743 INFO:teuthology.orchestra.run.smithi089.stdout:checking for FIEXCHANGE_RANGE... no 2023-07-06T02:24:41.774 INFO:teuthology.orchestra.run.smithi089.stdout:checking for renameat2... yes 2023-07-06T02:24:41.805 INFO:teuthology.orchestra.run.smithi089.stdout:checking for reallocarray... yes 2023-07-06T02:24:41.818 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct mount_attr... no 2023-07-06T02:24:41.857 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_qgroup_limit... yes 2023-07-06T02:24:41.890 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-07-06T02:24:41.923 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-07-06T02:24:41.956 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-07-06T02:24:41.989 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-07-06T02:24:42.022 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-07-06T02:24:42.055 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-07-06T02:24:42.080 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/btrfs.h usability... yes 2023-07-06T02:24:42.087 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/btrfs.h presence... yes 2023-07-06T02:24:42.087 INFO:teuthology.orchestra.run.smithi089.stdout:checking for linux/btrfs.h... yes 2023-07-06T02:24:42.116 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/btrfs_tree.h usability... yes 2023-07-06T02:24:42.123 INFO:teuthology.orchestra.run.smithi089.stdout:checking linux/btrfs_tree.h presence... yes 2023-07-06T02:24:42.124 INFO:teuthology.orchestra.run.smithi089.stdout:checking for linux/btrfs_tree.h... yes 2023-07-06T02:24:42.155 INFO:teuthology.orchestra.run.smithi089.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... no 2023-07-06T02:24:42.170 INFO:teuthology.orchestra.run.smithi089.stdout:configure: creating ./config.status 2023-07-06T02:24:42.456 INFO:teuthology.orchestra.run.smithi089.stdout:config.status: creating include/builddefs 2023-07-06T02:24:42.466 INFO:teuthology.orchestra.run.smithi089.stdout:config.status: creating include/config.h 2023-07-06T02:24:42.474 INFO:teuthology.orchestra.run.smithi089.stdout:config.status: executing libtool commands 2023-07-06T02:24:42.511 INFO:teuthology.orchestra.run.smithi089.stdout:Building include 2023-07-06T02:24:42.514 INFO:teuthology.orchestra.run.smithi089.stdout:Building lib 2023-07-06T02:24:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: [LTDEP] 2023-07-06T02:24:42.619 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] dataascii.lo 2023-07-06T02:24:42.730 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] databin.lo 2023-07-06T02:24:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] datapid.lo 2023-07-06T02:24:42.963 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] file_lock.lo 2023-07-06T02:24:43.099 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] forker.lo 2023-07-06T02:24:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] pattern.lo 2023-07-06T02:24:43.336 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] open_flags.lo 2023-07-06T02:24:43.466 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] random_range.lo 2023-07-06T02:24:43.671 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] string_to_tokens.lo 2023-07-06T02:24:43.767 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] str_to_bytes.lo 2023-07-06T02:24:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] tlibio.lo 2023-07-06T02:24:44.096 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] write_log.lo 2023-07-06T02:24:44.315 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] random.lo 2023-07-06T02:24:44.437 INFO:teuthology.orchestra.run.smithi089.stdout: [LD] 2023-07-06T02:24:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:Building ltp 2023-07-06T02:24:44.509 INFO:teuthology.orchestra.run.smithi089.stdout: [DEP] 2023-07-06T02:24:44.622 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] doio 2023-07-06T02:24:45.206 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fsstress 2023-07-06T02:24:46.386 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fsx 2023-07-06T02:24:47.161 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] growfiles 2023-07-06T02:24:47.617 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] iogen 2023-07-06T02:24:47.935 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-stress 2023-07-06T02:24:48.311 INFO:teuthology.orchestra.run.smithi089.stdout:Building src 2023-07-06T02:24:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: [DEP] 2023-07-06T02:24:49.638 INFO:teuthology.orchestra.run.smithi089.stdout: [DEP] 2023-07-06T02:24:49.663 INFO:teuthology.orchestra.run.smithi089.stdout: [DEP] 2023-07-06T02:24:49.670 INFO:teuthology.orchestra.run.smithi089.stdout: [DEP] 2023-07-06T02:24:49.847 INFO:teuthology.orchestra.run.smithi089.stdout: [DEP] 2023-07-06T02:24:49.868 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] dirstress 2023-07-06T02:24:50.080 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fill 2023-07-06T02:24:50.220 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fill2 2023-07-06T02:24:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] getpagesize 2023-07-06T02:24:50.548 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] holes 2023-07-06T02:24:50.724 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] lstat64 2023-07-06T02:24:50.890 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] nametest 2023-07-06T02:24:51.102 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] permname 2023-07-06T02:24:51.282 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] randholes 2023-07-06T02:24:51.503 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] runas 2023-07-06T02:24:51.661 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] truncfile 2023-07-06T02:24:51.822 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] usemem 2023-07-06T02:24:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] mmapcat 2023-07-06T02:24:52.093 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] append_reader 2023-07-06T02:24:52.231 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] append_writer 2023-07-06T02:24:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] dirperf 2023-07-06T02:24:52.565 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] metaperf 2023-07-06T02:24:52.792 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] devzero 2023-07-06T02:24:52.965 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] feature 2023-07-06T02:24:53.156 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] alloc 2023-07-06T02:24:53.345 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fault 2023-07-06T02:24:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fstest 2023-07-06T02:24:53.704 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_access_root 2023-07-06T02:24:53.844 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] godown 2023-07-06T02:24:53.997 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] resvtest 2023-07-06T02:24:54.163 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] writemod 2023-07-06T02:24:54.296 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] writev_on_pagefault 2023-07-06T02:24:54.439 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] makeextents 2023-07-06T02:24:54.600 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] itrash 2023-07-06T02:24:54.741 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] rename 2023-07-06T02:24:54.869 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] multi_open_unlink 2023-07-06T02:24:55.018 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] unwritten_sync 2023-07-06T02:24:55.180 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] genhashnames 2023-07-06T02:24:55.350 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_holes 2023-07-06T02:24:55.484 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_writev 2023-07-06T02:24:55.620 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_truncate_cmtime 2023-07-06T02:24:55.764 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] dirhash_collide 2023-07-06T02:24:55.928 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_rename_overwrite 2023-07-06T02:24:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] holetest 2023-07-06T02:24:56.254 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_truncate_self 2023-07-06T02:24:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] af_unix 2023-07-06T02:24:56.521 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_stale_pmd 2023-07-06T02:24:56.660 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_cow_race 2023-07-06T02:24:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_fallocate 2023-07-06T02:24:56.946 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fsync-err 2023-07-06T02:24:57.113 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_write_ro 2023-07-06T02:24:57.250 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_ext4_dax_journal_corruption 2023-07-06T02:24:57.395 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_ext4_dax_inline_corruption 2023-07-06T02:24:57.537 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_ofd_locks 2023-07-06T02:24:57.717 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_collision 2023-07-06T02:24:57.896 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] mmap-write-concurrent 2023-07-06T02:24:58.050 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_get_file_time 2023-07-06T02:24:58.184 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_create_short_dirs 2023-07-06T02:24:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_create_long_dirs 2023-07-06T02:24:58.485 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_enospc 2023-07-06T02:24:58.658 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_writev_overlap 2023-07-06T02:24:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] checkpoint_journal 2023-07-06T02:24:58.953 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] mmap-rw-fault 2023-07-06T02:24:59.017 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:24:59.019 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:24:59.145 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] allocstale 2023-07-06T02:24:59.298 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_cow_memory_failure 2023-07-06T02:24:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fake-dump-rootino 2023-07-06T02:24:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] dio-buf-fault 2023-07-06T02:24:59.730 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] xfsctl 2023-07-06T02:24:59.899 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] bstat 2023-07-06T02:25:00.082 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mtab 2023-07-06T02:25:00.249 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] getdevicesize 2023-07-06T02:25:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] preallo_rw_pattern_reader 2023-07-06T02:25:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] preallo_rw_pattern_writer 2023-07-06T02:25:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] ftrunc 2023-07-06T02:25:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] trunc 2023-07-06T02:25:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fs_perms 2023-07-06T02:25:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] testx 2023-07-06T02:25:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] looptest 2023-07-06T02:25:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] locktest 2023-07-06T02:25:01.667 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] unwritten_mmap 2023-07-06T02:25:01.815 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] bulkstat_unlink_test 2023-07-06T02:25:01.996 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] deduperace 2023-07-06T02:25:02.171 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] bulkstat_unlink_test_modified 2023-07-06T02:25:02.348 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_dir_offset 2023-07-06T02:25:02.483 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_futimens 2023-07-06T02:25:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_immutable 2023-07-06T02:25:03.154 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] stale_handle 2023-07-06T02:25:03.309 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] pwrite_mmap_blocked 2023-07-06T02:25:03.448 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_dir_offset2 2023-07-06T02:25:03.622 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] seek_sanity_test 2023-07-06T02:25:03.993 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] seek_copy_test 2023-07-06T02:25:04.194 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_readdir_1 2023-07-06T02:25:04.329 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_readdir_2 2023-07-06T02:25:04.467 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fsync-tester 2023-07-06T02:25:04.691 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] nsexec 2023-07-06T02:25:04.851 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] cloner 2023-07-06T02:25:05.016 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] renameat2 2023-07-06T02:25:05.163 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_getcwd 2023-07-06T02:25:05.307 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] e4compact 2023-07-06T02:25:05.487 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] test-nextquota 2023-07-06T02:25:05.642 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] punch-alternating 2023-07-06T02:25:05.797 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] attr-list-by-handle-cursor-test 2023-07-06T02:25:05.948 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] listxattr 2023-07-06T02:25:06.083 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] dio-interleaved 2023-07-06T02:25:06.230 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_dir_type 2023-07-06T02:25:06.373 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] dio-invalidate-cache 2023-07-06T02:25:06.556 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] stat_test 2023-07-06T02:25:06.824 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_encrypted_d_revalidate 2023-07-06T02:25:06.975 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] attr_replace_test 2023-07-06T02:25:07.122 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] swapon 2023-07-06T02:25:07.272 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] mkswap 2023-07-06T02:25:07.411 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_attr_corruption 2023-07-06T02:25:07.548 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_open_tmpfiles 2023-07-06T02:25:07.710 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fscrypt-crypt-util 2023-07-06T02:25:08.277 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] bulkstat_null_ocount 2023-07-06T02:25:08.421 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] splice-test 2023-07-06T02:25:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] chprojid_fail 2023-07-06T02:25:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] detached_mounts_propagation 2023-07-06T02:25:08.898 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] ext4_resize 2023-07-06T02:25:09.034 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_readdir_3 2023-07-06T02:25:09.209 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] splice2pipe 2023-07-06T02:25:09.359 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] uuid_ioctl 2023-07-06T02:25:09.500 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fiemap-tester 2023-07-06T02:25:09.739 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_stripealign 2023-07-06T02:25:09.930 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] open_by_handle 2023-07-06T02:25:10.160 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] t_mmap_dio 2023-07-06T02:25:10.308 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] xfsfind 2023-07-06T02:25:10.482 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] fssum 2023-07-06T02:25:10.869 INFO:teuthology.orchestra.run.smithi089.stdout:Building log-writes 2023-07-06T02:25:10.874 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] replay-log 2023-07-06T02:25:11.151 INFO:teuthology.orchestra.run.smithi089.stdout:Building perf 2023-07-06T02:25:11.155 INFO:teuthology.orchestra.run.smithi089.stdout:Building aio-dio-regress 2023-07-06T02:25:11.160 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-append-write-fallocate-race 2023-07-06T02:25:11.320 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-fcntl-race 2023-07-06T02:25:11.471 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-subblock-eof-read 2023-07-06T02:25:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-append-write-read-race 2023-07-06T02:25:11.767 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-invalidate-readahead 2023-07-06T02:25:11.920 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-write-verify 2023-07-06T02:25:12.096 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-07-06T02:25:12.224 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-last-ref-held-by-io 2023-07-06T02:25:12.385 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-cycle-write 2023-07-06T02:25:12.555 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-extend-stat 2023-07-06T02:25:12.708 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-07-06T02:25:12.832 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-hole-filling-race 2023-07-06T02:25:12.972 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-invalidate-failure 2023-07-06T02:25:13.120 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aiodio_sparse2 2023-07-06T02:25:13.318 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aiocp 2023-07-06T02:25:13.523 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-cow-race 2023-07-06T02:25:13.667 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] aio-dio-eof-race 2023-07-06T02:25:13.836 INFO:teuthology.orchestra.run.smithi089.stdout:Building vfs 2023-07-06T02:25:13.841 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] vfstest 2023-07-06T02:25:15.503 INFO:teuthology.orchestra.run.smithi089.stdout: [CC] mount-idmapped 2023-07-06T02:25:15.834 INFO:teuthology.orchestra.run.smithi089.stdout:Building m4 2023-07-06T02:25:15.837 INFO:teuthology.orchestra.run.smithi089.stdout:Building common 2023-07-06T02:25:15.840 INFO:teuthology.orchestra.run.smithi089.stdout:Building tests 2023-07-06T02:25:15.845 INFO:teuthology.orchestra.run.smithi089.stdout:Building perf 2023-07-06T02:25:15.850 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/perf/group.list 2023-07-06T02:25:15.860 INFO:teuthology.orchestra.run.smithi089.stdout:Building ext4 2023-07-06T02:25:15.865 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ext4/group.list 2023-07-06T02:25:15.899 INFO:teuthology.orchestra.run.smithi089.stdout:Building ocfs2 2023-07-06T02:25:15.904 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ocfs2/group.list 2023-07-06T02:25:15.915 INFO:teuthology.orchestra.run.smithi089.stdout:Building overlay 2023-07-06T02:25:15.920 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/overlay/group.list 2023-07-06T02:25:15.954 INFO:teuthology.orchestra.run.smithi089.stdout:Building cifs 2023-07-06T02:25:15.959 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/cifs/group.list 2023-07-06T02:25:15.970 INFO:teuthology.orchestra.run.smithi089.stdout:Building generic 2023-07-06T02:25:15.976 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/generic/group.list 2023-07-06T02:25:16.064 INFO:teuthology.orchestra.run.smithi089.stdout:Building shared 2023-07-06T02:25:16.069 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/shared/group.list 2023-07-06T02:25:16.084 INFO:teuthology.orchestra.run.smithi089.stdout:Building nfs 2023-07-06T02:25:16.089 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/nfs/group.list 2023-07-06T02:25:16.102 INFO:teuthology.orchestra.run.smithi089.stdout:Building f2fs 2023-07-06T02:25:16.107 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/f2fs/group.list 2023-07-06T02:25:16.122 INFO:teuthology.orchestra.run.smithi089.stdout:Building udf 2023-07-06T02:25:16.126 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/udf/group.list 2023-07-06T02:25:16.137 INFO:teuthology.orchestra.run.smithi089.stdout:Building ceph 2023-07-06T02:25:16.142 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ceph/group.list 2023-07-06T02:25:16.154 INFO:teuthology.orchestra.run.smithi089.stdout:Building xfs 2023-07-06T02:25:16.160 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/xfs/group.list 2023-07-06T02:25:16.250 INFO:teuthology.orchestra.run.smithi089.stdout:Building btrfs 2023-07-06T02:25:16.255 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/btrfs/group.list 2023-07-06T02:25:16.307 INFO:teuthology.orchestra.run.smithi089.stdout:Building tmpfs 2023-07-06T02:25:16.312 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/tmpfs/group.list 2023-07-06T02:25:16.323 INFO:teuthology.orchestra.run.smithi089.stdout:Building selftest 2023-07-06T02:25:16.328 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/selftest/group.list 2023-07-06T02:25:16.341 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo make install) 2023-07-06T02:25:16.378 INFO:teuthology.orchestra.run.smithi089.stdout:Building include 2023-07-06T02:25:16.381 INFO:teuthology.orchestra.run.smithi089.stdout:Building lib 2023-07-06T02:25:16.384 INFO:teuthology.orchestra.run.smithi089.stdout:Building ltp 2023-07-06T02:25:16.388 INFO:teuthology.orchestra.run.smithi089.stdout:Building src 2023-07-06T02:25:16.421 INFO:teuthology.orchestra.run.smithi089.stdout:Building log-writes 2023-07-06T02:25:16.424 INFO:teuthology.orchestra.run.smithi089.stdout:Building perf 2023-07-06T02:25:16.428 INFO:teuthology.orchestra.run.smithi089.stdout:Building aio-dio-regress 2023-07-06T02:25:16.431 INFO:teuthology.orchestra.run.smithi089.stdout:Building vfs 2023-07-06T02:25:16.435 INFO:teuthology.orchestra.run.smithi089.stdout:Building m4 2023-07-06T02:25:16.438 INFO:teuthology.orchestra.run.smithi089.stdout:Building common 2023-07-06T02:25:16.442 INFO:teuthology.orchestra.run.smithi089.stdout:Building tests 2023-07-06T02:25:16.447 INFO:teuthology.orchestra.run.smithi089.stdout:Building perf 2023-07-06T02:25:16.451 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/perf/group.list 2023-07-06T02:25:16.462 INFO:teuthology.orchestra.run.smithi089.stdout:Building ext4 2023-07-06T02:25:16.467 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ext4/group.list 2023-07-06T02:25:16.500 INFO:teuthology.orchestra.run.smithi089.stdout:Building ocfs2 2023-07-06T02:25:16.505 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ocfs2/group.list 2023-07-06T02:25:16.517 INFO:teuthology.orchestra.run.smithi089.stdout:Building overlay 2023-07-06T02:25:16.522 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/overlay/group.list 2023-07-06T02:25:16.555 INFO:teuthology.orchestra.run.smithi089.stdout:Building cifs 2023-07-06T02:25:16.560 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/cifs/group.list 2023-07-06T02:25:16.571 INFO:teuthology.orchestra.run.smithi089.stdout:Building generic 2023-07-06T02:25:16.577 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/generic/group.list 2023-07-06T02:25:16.663 INFO:teuthology.orchestra.run.smithi089.stdout:Building shared 2023-07-06T02:25:16.668 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/shared/group.list 2023-07-06T02:25:16.682 INFO:teuthology.orchestra.run.smithi089.stdout:Building nfs 2023-07-06T02:25:16.687 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/nfs/group.list 2023-07-06T02:25:16.700 INFO:teuthology.orchestra.run.smithi089.stdout:Building f2fs 2023-07-06T02:25:16.704 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/f2fs/group.list 2023-07-06T02:25:16.720 INFO:teuthology.orchestra.run.smithi089.stdout:Building udf 2023-07-06T02:25:16.724 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/udf/group.list 2023-07-06T02:25:16.735 INFO:teuthology.orchestra.run.smithi089.stdout:Building ceph 2023-07-06T02:25:16.740 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ceph/group.list 2023-07-06T02:25:16.752 INFO:teuthology.orchestra.run.smithi089.stdout:Building xfs 2023-07-06T02:25:16.761 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/xfs/group.list 2023-07-06T02:25:16.846 INFO:teuthology.orchestra.run.smithi089.stdout:Building btrfs 2023-07-06T02:25:16.851 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/btrfs/group.list 2023-07-06T02:25:16.903 INFO:teuthology.orchestra.run.smithi089.stdout:Building tmpfs 2023-07-06T02:25:16.907 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/tmpfs/group.list 2023-07-06T02:25:16.919 INFO:teuthology.orchestra.run.smithi089.stdout:Building selftest 2023-07-06T02:25:16.924 INFO:teuthology.orchestra.run.smithi089.stdout: [GROUP] /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/selftest/group.list 2023-07-06T02:25:16.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2023-07-06T02:25:16.936 INFO:teuthology.orchestra.run.smithi089.stdout:gmake[1]: Nothing to be done for 'install'. 2023-07-06T02:25:16.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2023-07-06T02:25:16.939 INFO:teuthology.orchestra.run.smithi089.stdout:gmake[1]: Nothing to be done for 'install'. 2023-07-06T02:25:16.939 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2023-07-06T02:25:16.941 INFO:teuthology.orchestra.run.smithi089.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-07-06T02:25:16.949 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-07-06T02:25:17.025 INFO:teuthology.orchestra.run.smithi089.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-07-06T02:25:17.033 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2023-07-06T02:25:17.049 INFO:teuthology.orchestra.run.smithi089.stdout:Building log-writes 2023-07-06T02:25:17.053 INFO:teuthology.orchestra.run.smithi089.stdout:Building perf 2023-07-06T02:25:17.056 INFO:teuthology.orchestra.run.smithi089.stdout:Building aio-dio-regress 2023-07-06T02:25:17.060 INFO:teuthology.orchestra.run.smithi089.stdout:Building vfs 2023-07-06T02:25:17.062 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake -C log-writes install 2023-07-06T02:25:17.064 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-07-06T02:25:17.073 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-07-06T02:25:17.081 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake -C perf install 2023-07-06T02:25:17.082 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-07-06T02:25:17.091 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf 2023-07-06T02:25:17.114 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake -C aio-dio-regress install 2023-07-06T02:25:17.115 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-07-06T02:25:17.124 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 aio-dio-append-write-fallocate-race aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-hole-filling-race aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress 2023-07-06T02:25:17.177 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake -C vfs install 2023-07-06T02:25:17.178 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-07-06T02:25:17.186 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-07-06T02:25:17.198 INFO:teuthology.orchestra.run.smithi089.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-07-06T02:25:17.207 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-07-06T02:25:18.388 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py /var/lib/xfstests/src 2023-07-06T02:25:18.480 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-07-06T02:25:18.510 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2023-07-06T02:25:18.512 INFO:teuthology.orchestra.run.smithi089.stdout:gmake[1]: Nothing to be done for 'install'. 2023-07-06T02:25:18.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2023-07-06T02:25:18.513 INFO:teuthology.orchestra.run.smithi089.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-07-06T02:25:18.522 INFO:teuthology.orchestra.run.smithi089.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-07-06T02:25:18.653 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2023-07-06T02:25:18.655 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/btrfs/ install 2023-07-06T02:25:18.657 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-07-06T02:25:18.666 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 290 291 299 /var/lib/xfstests/tests/btrfs 2023-07-06T02:25:19.456 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-07-06T02:25:19.465 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 057.out 175.out 186.out 240.out 069.out 092.out 077.out 140.out 265.out 236.out 108.out 204.out 177.out 194.out 234.out 249.out 109.out 033.out 082.out 252.out 270.out 040.out 176.out 202.out 137.out 034.out 125.out 014.out 165.out 181.out 152.out 142.out 264.out 192.out 148.out 282.out 267.out 052.out 228.out 183.out 094.out 273.out 251.out 110.out 043.out 209.out 151.out 206.out 195.out 274.out 215.out 115.out 015.out 250.out 191.out 281.out 046.out 051.out 244.out 219.out 119.out 019.out 128.out 232.out 248.out 162.out 147.out 197.out 196.out 242.out 004.out 189.out 220.out 120.out 020.out 102.out 263.out 171.out 261.out 045.out 217.out 117.out 017.out 072.out 068.out 048.out 062.out 038.out 084.out 129.out 233.out 066.out 029.out 229.out 103.out 254.out 111.out 011.out 243.out 275.out 134.out 170.out 260.out 157.out 143.out 133.out 286.out 161.out 246.out 216.out 116.out 016.out 155.out 118.out 287.out 067.out 010.out 238.out 138.out 095.out 184.out 160.out 190.out 291.out 081.out 024.out 071.out 061.out 047.out 200.out 100.out 025.out 139.out 258.out 185.out 269.out 208.out 039.out 222.out 035.out 169.out 098.out 290.out 080.out 070.out 060.out 207.out 106.out 009.out 167.out 003.out 211.out 158.out 085.out 146.out 188.out 104.out 089.out 086.out 279.out 225.out 174.out 156.out 135.out 257.out 124.out 172.out 058.out 262.out 036.out 042.out 193.out 097.out 271.out 283.out 083.out 199.out 099.out 278.out 053.out 164.out 239.out 241.out 002.out 178.out 141.out 214.out 114.out 074.out 056.out 012.out 041.out 223.out 150.out 123.out 179.out 144.out 032.out 277.out 205.out 136.out 005.out 096.out 198.out 008.out 280.out 050.out 105.out 027.out 187.out 091.out 153.out 088.out 285.out 044.out 201.out 101.out 001.out 266.out 237.out 065.out 231.out 227.out 075.out 131.out 213.out 113.out 013.out 031.out 182.out 272.out 078.out 173.out 259.out 210.out 132.out 037.out 093.out 235.out 127.out 230.out 166.out 130.out 023.out 073.out 030.out 203.out 063.out 055.out 245.out 145.out 159.out 122.out 076.out 022.out 006.out 284.out 299.out 224.out 154.out 276.out 149.out 064.out 218.out 018.out 168.out 007.out 079.out 054.out 059.out 049.out 226.out 163.out 126.out 121.out 026.out 180.out 268.out 253.out 255.out 090.out 221.out 087.out 212.out 112.out 256.out 021.out 247.out 028.out 107.out /var/lib/xfstests/tests/btrfs 2023-07-06T02:25:20.256 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ceph/ install 2023-07-06T02:25:20.257 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-07-06T02:25:20.266 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-07-06T02:25:20.286 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-07-06T02:25:20.294 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/ceph 2023-07-06T02:25:20.313 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/cifs/ install 2023-07-06T02:25:20.315 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-07-06T02:25:20.323 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-07-06T02:25:20.332 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-07-06T02:25:20.340 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-07-06T02:25:20.349 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ext4/ install 2023-07-06T02:25:20.351 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-07-06T02:25:20.359 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-07-06T02:25:20.546 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-07-06T02:25:20.554 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 001.out.nodelalloc 001.out.default 001.out.nozero 001.cfg 032.out 039.out 305.out 005.out 008.out 035.out 036.out 308.out 050.out 040.out 306.out 007.out 021.out 046.out 012.out 009.out 033.out 045.out 034.out 056.out 047.out 018.out 023.out 302.out 020.out 002.out 014.out 024.out 013.out 031.out 015.out 304.out 004.out 048.out 038.out 052.out 303.out 029.out 011.out 027.out 042.out 044.out 055.out 030.out 058.out 003.out 017.out 043.out 053.out 016.out 059.out 037.out 301.out 010.out 022.out 006.out 057.out 025.out 026.out 271.out 051.out 041.out 307.out 019.out 028.out 054.out 049.out /var/lib/xfstests/tests/ext4 2023-07-06T02:25:20.752 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/f2fs/ install 2023-07-06T02:25:20.753 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-07-06T02:25:20.762 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-07-06T02:25:20.774 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-07-06T02:25:20.782 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-07-06T02:25:20.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/generic/ install 2023-07-06T02:25:20.798 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-07-06T02:25:20.807 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 /var/lib/xfstests/tests/generic 2023-07-06T02:25:22.773 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-07-06T02:25:22.782 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 050.out.xfsquota 035.out.nfs 035.out.default 050.out.nojournal 050.out.default 050.cfg 035.cfg 208.out 433.out 670.out 040.out 104.out 575.out 537.out 683.out 497.out 337.out 702.out 460.out 149.out 088.out 366.out 202.out 137.out 034.out 110.out 389.out 403.out 593.out 571.out 243.out 592.out 282.out 472.out 468.out 628.out 528.out 607.out 428.out 659.out 407.out 228.out 128.out 171.out 028.out 620.out 273.out 474.out 492.out 609.out 707.out 693.out 595.out 506.out 268.out 632.out 532.out 591.out 281.out 471.out 619.out 519.out 013.out 419.out 319.out 177.out 523.out 119.out 019.out 727.out 372.out 686.out 312.out 718.out 143.out 716.out 161.out 516.out 518.out 538.out 655.out 216.out 116.out 016.out 155.out 338.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 457.out 333.out 258.out 362.out 379.out 691.out 039.out 173.out 498.out 665.out 098.out 711.out 290.out 480.out 706.out 060.out 406.out 306.out 106.out 642.out 447.out 217.out 120.out 191.out 411.out 357.out 481.out 211.out 158.out 014.out 182.out 370.out 188.out 030.out 099.out 585.out 724.out 721.out 624.out 524.out 174.out 156.out 324.out 224.out 313.out 355.out 124.out 382.out 672.out 278.out 063.out 058.out 384.out 296.out 042.out 193.out 271.out 465.out 473.out 335.out 247.out 509.out 568.out 441.out 440.out 475.out 548.out 356.out 341.out 094.out 241.out 600.out 089.out 714.out 614.out 514.out 414.out 314.out 416.out 214.out 381.out 056.out 041.out 057.out 340.out 358.out 572.out 586.out 521.out 240.out 549.out 095.out 633.out 434.out 140.out 708.out 533.out 508.out 408.out 308.out 476.out 087.out 199.out 458.out 233.out 639.out 443.out 540.out 397.out 053.out 279.out 276.out 448.out 164.out 376.out 589.out 587.out 339.out 294.out 352.out 484.out 489.out 139.out 317.out 399.out 220.out 552.out 695.out 017.out 677.out 097.out 175.out 315.out 295.out 580.out 075.out 186.out 452.out 679.out 292.out 252.out 077.out 167.out 688.out 206.out 348.out 394.out 651.out 184.out 466.out 697.out 495.out 725.out 109.out 482.out 625.out 283.out 525.out 488.out 496.out 325.out 551.out 225.out 322.out 636.out 125.out 715.out 192.out 544.out 515.out 650.out 152.out 658.out 563.out 393.out 578.out 015.out 392.out 682.out 684.out 550.out 052.out 652.out 534.out 351.out 583.out 451.out 450.out 463.out 251.out 267.out 176.out 069.out 644.out 065.out 487.out 615.out 395.out 676.out 415.out 344.out 215.out 115.out 444.out 027.out 250.out 391.out 674.out 464.out 705.out 536.out 577.out 439.out 244.out 336.out 144.out 505.out 405.out 342.out 205.out 136.out 598.out 363.out 036.out 198.out 723.out 008.out 390.out 680.out 605.out 305.out 105.out 005.out 367.out 291.out 719.out 323.out 685.out 562.out 570.out 388.out 527.out 237.out 151.out 227.out 163.out 332.out 661.out 582.out 232.out 478.out 584.out 462.out 597.out 412.out 132.out 179.out 037.out 293.out 499.out 638.out 353.out 020.out 660.out 561.out 141.out 076.out 668.out 044.out 648.out 387.out 594.out 438.out 265.out 289.out 574.out 006.out 690.out 284.out 567.out 299.out 456.out 461.out 477.out 307.out 647.out 493.out 107.out 007.out 380.out 526.out 479.out 426.out 150.out 326.out 082.out 226.out 413.out 126.out 025.out 121.out 026.out 021.out 485.out 694.out 347.out 275.out 090.out 361.out 564.out 302.out 432.out 002.out 558.out 162.out 147.out 288.out 197.out 033.out 396.out 051.out 350.out 547.out 047.out 024.out 360.out 543.out 261.out 045.out 692.out 255.out 726.out 378.out 346.out 048.out 062.out 626.out 529.out 231.out 343.out 662.out 425.out 329.out 165.out 334.out 103.out 003.out 129.out 423.out 371.out 234.out 183.out 671.out 645.out 673.out 190.out 486.out 223.out 123.out 262.out 669.out 023.out 459.out 616.out 032.out 266.out 146.out 398.out 728.out 696.out 427.out 590.out 280.out 470.out 455.out 187.out 091.out 046.out 374.out 631.out 328.out 260.out 654.out 359.out 531.out 649.out 301.out 101.out 635.out 623.out 385.out 331.out 556.out 713.out 613.out 513.out 622.out 131.out 213.out 634.out 031.out 565.out 630.out 675.out 272.out 530.out 510.out 627.out 573.out 576.out 430.out 310.out 569.out 454.out 259.out 210.out 449.out 327.out 330.out 067.out 093.out 604.out 287.out 127.out 699.out 166.out 130.out 219.out 073.out 603.out 503.out 303.out 203.out 354.out 157.out 386.out 349.out 055.out 245.out 709.out 522.out 422.out 222.out 159.out 539.out 249.out 122.out 689.out 022.out 681.out 424.out 321.out 066.out 490.out 435.out 375.out 134.out 154.out 566.out 618.out 501.out 418.out 318.out 542.out 218.out 118.out 018.out 377.out 172.out 168.out 079.out 469.out 054.out 059.out 049.out 442.out 657.out 621.out 180.out 270.out 253.out 421.out 712.out 612.out 667.out 512.out 494.out 221.out 212.out 112.out 256.out 012.out 029.out 195.out 554.out 148.out 185.out 596.out 720.out 248.out 196.out 520.out 242.out 420.out 285.out 320.out 629.out 230.out 602.out 502.out 402.out 102.out 717.out 617.out 517.out 429.out 417.out 365.out 092.out 117.out 664.out 072.out 068.out 229.out 038.out 084.out 369.out 611.out 511.out 311.out 368.out 637.out 111.out 011.out 257.out 194.out 170.out 606.out 133.out 286.out 043.out 437.out 545.out 559.out 710.out 610.out 687.out 410.out 701.out 601.out 401.out 201.out 010.out 001.out 096.out 114.out 113.out 074.out 581.out 145.out 641.out 491.out 081.out 071.out 083.out 653.out 700.out 064.out 400.out 300.out 200.out 100.out 646.out 541.out 445.out 269.out 546.out 656.out 698.out 553.out 178.out 298.out 446.out 080.out 070.out 608.out 507.out 409.out 309.out 209.out 108.out 009.out 643.out 004.out 557.out 453.out 666.out 236.out 078.out 364.out 555.out 383.out 085.out 704.out 588.out 504.out 404.out 304.out 204.out 535.out 345.out 246.out 467.out 086.out 254.out 207.out 142.out 235.out 436.out 703.out 135.out 264.out 431.out 678.out 373.out 316.out 169.out 640.out 579.out 297.out 153.out 560.out 663.out 599.out /var/lib/xfstests/tests/generic 2023-07-06T02:25:24.779 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/nfs/ install 2023-07-06T02:25:24.781 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-07-06T02:25:24.789 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-07-06T02:25:24.798 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-07-06T02:25:24.806 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-07-06T02:25:24.815 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/ocfs2/ install 2023-07-06T02:25:24.822 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-07-06T02:25:24.825 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-07-06T02:25:24.834 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-07-06T02:25:24.842 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-07-06T02:25:24.851 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/overlay/ install 2023-07-06T02:25:24.853 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-07-06T02:25:24.862 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-07-06T02:25:25.126 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-07-06T02:25:25.134 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 019.out 034.out 018.out 024.out 077.out 069.out 020.out 102.out 014.out 013.out 031.out 015.out 117.out 017.out 072.out 068.out 063.out 048.out 062.out 038.out 026.out 042.out 052.out 110.out 067.out 029.out 111.out 055.out 112.out 073.out 078.out 030.out 058.out 003.out 045.out 023.out 043.out 053.out 109.out 036.out 116.out 016.out 065.out 010.out 001.out 022.out 106.out 007.out 006.out 115.out 057.out 113.out 114.out 074.out 046.out 071.out 061.out 025.out 051.out 011.out 041.out 047.out 100.out 064.out 107.out 104.out 028.out 076.out 054.out 059.out 008.out 049.out 032.out 039.out 021.out 005.out 002.out 035.out 108.out 070.out 060.out 050.out 027.out 105.out 009.out 004.out 037.out 044.out 012.out 101.out 103.out 033.out 066.out 040.out 056.out 075.out /var/lib/xfstests/tests/overlay 2023-07-06T02:25:25.399 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/perf/ install 2023-07-06T02:25:25.400 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-07-06T02:25:25.409 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-07-06T02:25:25.418 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-07-06T02:25:25.426 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-07-06T02:25:25.435 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/selftest/ install 2023-07-06T02:25:25.436 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-07-06T02:25:25.445 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-07-06T02:25:25.467 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-07-06T02:25:25.476 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 006.out 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/selftest 2023-07-06T02:25:25.498 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/shared/ install 2023-07-06T02:25:25.499 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-07-06T02:25:25.508 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-07-06T02:25:25.522 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-07-06T02:25:25.530 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 032.out 298.out 002.out /var/lib/xfstests/tests/shared 2023-07-06T02:25:25.544 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/tmpfs/ install 2023-07-06T02:25:25.546 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-07-06T02:25:25.554 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-07-06T02:25:25.563 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-07-06T02:25:25.572 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-07-06T02:25:25.580 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/udf/ install 2023-07-06T02:25:25.582 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-07-06T02:25:25.590 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-07-06T02:25:25.599 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-07-06T02:25:25.607 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-07-06T02:25:25.616 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.KkaxlHRTc5xfstests-dev/tests/xfs/ install 2023-07-06T02:25:25.620 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-07-06T02:25:25.629 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-07-06T02:25:27.491 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-07-06T02:25:27.500 INFO:teuthology.orchestra.run.smithi089.stdout:../../install-sh -o root -g root -m 644 096.out.internal 033.out.crc 096.out.external 033.out.default 071.out.64 216.out.classic 216.out.64mblog 071.out.32 216.cfg 033.cfg 040.out 304.out 349.out 457.out 537.out 295.out 497.out 337.out 385.out 082.out 270.out 460.out 149.out 759.out 502.out 088.out 366.out 202.out 137.out 746.out 110.out 468.out 018.out 593.out 571.out 567.out 592.out 282.out 267.out 728.out 178.out 799.out 528.out 769.out 428.out 507.out 407.out 228.out 128.out 171.out 183.out 247.out 273.out 474.out 109.out 595.out 506.out 732.out 268.out 035.out 113.out 532.out 591.out 281.out 471.out 723.out 519.out 319.out 467.out 523.out 119.out 775.out 727.out 372.out 503.out 526.out 329.out 165.out 229.out 103.out 129.out 371.out 234.out 738.out 078.out 718.out 143.out 716.out 161.out 516.out 416.out 116.out 043.out 155.out 338.out 067.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 047.out 772.out 258.out 379.out 322.out 039.out 173.out 222.out 742.out 098.out 426.out 290.out 480.out 060.out 406.out 306.out 141.out 106.out 006.out 167.out 191.out 017.out 542.out 211.out 783.out 485.out 442.out 099.out 177.out 174.out 156.out 771.out 324.out 224.out 313.out 124.out 758.out 278.out 058.out 741.out 384.out 773.out 042.out 193.out 392.out 768.out 399.out 473.out 770.out 709.out 755.out 364.out 209.out 441.out 440.out 157.out 740.out 356.out 197.out 341.out 187.out 094.out 241.out 501.out 089.out 540.out 365.out 514.out 414.out 314.out 214.out 381.out 056.out 041.out 340.out 793.out 358.out 572.out 568.out 586.out 240.out 549.out 092.out 439.out 434.out 140.out 708.out 265.out 508.out 408.out 308.out 208.out 737.out 204.out 535.out 104.out 733.out 544.out 059.out 249.out 335.out 142.out 235.out 575.out 253.out 715.out 533.out 264.out 431.out 784.out 433.out 316.out 169.out 579.out 297.out 393.out 333.out 476.out 087.out 199.out 739.out 068.out 233.out 752.out 736.out 443.out 086.out 053.out 749.out 279.out 276.out 711.out 164.out 589.out 587.out 339.out 577.out 294.out 352.out 458.out 498.out 489.out 139.out 774.out 790.out 552.out 781.out 357.out 065.out 186.out 452.out 069.out 252.out 585.out 751.out 724.out 206.out 190.out 794.out 348.out 184.out 435.out 466.out 248.out 725.out 785.out 482.out 750.out 525.out 496.out 325.out 176.out 225.out 125.out 789.out 192.out 296.out 152.out 563.out 246.out 551.out 436.out 792.out 075.out 472.out 578.out 463.out 550.out 052.out 534.out 351.out 430.out 583.out 305.out 451.out 425.out 450.out 744.out 251.out 444.out 487.out 762.out 465.out 395.out 515.out 415.out 389.out 315.out 215.out 115.out 015.out 027.out 250.out 344.out 046.out 051.out 536.out 254.out 030.out 244.out 336.out 028.out 150.out 144.out 505.out 342.out 136.out 005.out 036.out 198.out 008.out 390.out 405.out 588.out 205.out 105.out 367.out 291.out 437.out 719.out 323.out 363.out 562.out 382.out 570.out 388.out 527.out 158.out 766.out 019.out 237.out 151.out 760.out 227.out 163.out 332.out 796.out 232.out 478.out 584.out 462.out 220.out 412.out 132.out 179.out 037.out 293.out 492.out 499.out 112.out 748.out 020.out 786.out 063.out 561.out 076.out 044.out 014.out 362.out 387.out 594.out 438.out 721.out 289.out 574.out 284.out 548.out 299.out 419.out 521.out 581.out 461.out 477.out 307.out 726.out 107.out 007.out 448.out 262.out 380.out 093.out 479.out 547.out 326.out 226.out 345.out 126.out 025.out 121.out 447.out 761.out 580.out 013.out 347.out 275.out 090.out 361.out 564.out 788.out 066.out 455.out 432.out 747.out 558.out 162.out 147.out 288.out 050.out 396.out 576.out 350.out 354.out 298.out 301.out 360.out 543.out 261.out 734.out 045.out 292.out 255.out 729.out 378.out 346.out 048.out 062.out 529.out 231.out 343.out 486.out 223.out 123.out 023.out 459.out 032.out 798.out 146.out 398.out 168.out 427.out 590.out 280.out 470.out 484.out 491.out 767.out 016.out 488.out 731.out 782.out 374.out 285.out 127.out 328.out 260.out 359.out 531.out 376.out 101.out 266.out 331.out 791.out 743.out 556.out 713.out 024.out 513.out 131.out 213.out 730.out 095.out 031.out 565.out 182.out 272.out 530.out 754.out 573.out 454.out 259.out 449.out 327.out 330.out 777.out 493.out 010.out 287.out 230.out 166.out 130.out 219.out 073.out 370.out 135.out 303.out 203.out 003.out 355.out 386.out 560.out 055.out 245.out 243.out 522.out 787.out 745.out 422.out 394.out 159.out 539.out 524.out 122.out 022.out 424.out 779.out 490.out 795.out 353.out 423.out 134.out 154.out 764.out 271.out 518.out 418.out 318.out 077.out 218.out 118.out 757.out 377.out 172.out 034.out 079.out 469.out 054.out 049.out 765.out 021.out 002.out 456.out 180.out 421.out 712.out 403.out 321.out 057.out 512.out 494.out 221.out 312.out 212.out 538.out 256.out 012.out 195.out 283.out 554.out 397.out 185.out 373.out 720.out 481.out 196.out 520.out 242.out 026.out 420.out 188.out 320.out 555.out 120.out 153.out 402.out 302.out 102.out 717.out 517.out 429.out 417.out 317.out 217.out 117.out 475.out 072.out 778.out 097.out 038.out 084.out 369.out 511.out 148.out 411.out 029.out 311.out 368.out 111.out 011.out 257.out 194.out 170.out 133.out 286.out 756.out 175.out 545.out 714.out 559.out 710.out 510.out 413.out 410.out 310.out 210.out 401.out 201.out 001.out 495.out 114.out 753.out 074.out 145.out 797.out 464.out 091.out 081.out 083.out 800.out 064.out 400.out 300.out 200.out 100.out 375.out 541.out 445.out 269.out 546.out 780.out 553.out 763.out 446.out 080.out 070.out 509.out 409.out 309.out 207.out 108.out 009.out 391.out 004.out 557.out 453.out 236.out 566.out 582.out 735.out 383.out 085.out 334.out 504.out 404.out 776.out /var/lib/xfstests/tests/xfs 2023-07-06T02:25:29.170 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:25:29.173 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:25:29.384 INFO:teuthology.orchestra.run.smithi089.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-07-06T02:25:29.392 INFO:teuthology.orchestra.run.smithi089.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-07-06T02:25:29.401 INFO:teuthology.orchestra.run.smithi089.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-07-06T02:25:29.409 INFO:teuthology.orchestra.run.smithi089.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-07-06T02:25:29.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:25:29.421 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec ls tests/generic) 2023-07-06T02:25:29.439 INFO:teuthology.orchestra.run.smithi089.stdout:001 2023-07-06T02:25:29.439 INFO:teuthology.orchestra.run.smithi089.stdout:001.out 2023-07-06T02:25:29.439 INFO:teuthology.orchestra.run.smithi089.stdout:002 2023-07-06T02:25:29.439 INFO:teuthology.orchestra.run.smithi089.stdout:002.out 2023-07-06T02:25:29.440 INFO:teuthology.orchestra.run.smithi089.stdout:003 2023-07-06T02:25:29.440 INFO:teuthology.orchestra.run.smithi089.stdout:003.out 2023-07-06T02:25:29.440 INFO:teuthology.orchestra.run.smithi089.stdout:004 2023-07-06T02:25:29.440 INFO:teuthology.orchestra.run.smithi089.stdout:004.out 2023-07-06T02:25:29.440 INFO:teuthology.orchestra.run.smithi089.stdout:005 2023-07-06T02:25:29.440 INFO:teuthology.orchestra.run.smithi089.stdout:005.out 2023-07-06T02:25:29.440 INFO:teuthology.orchestra.run.smithi089.stdout:006 2023-07-06T02:25:29.441 INFO:teuthology.orchestra.run.smithi089.stdout:006.out 2023-07-06T02:25:29.441 INFO:teuthology.orchestra.run.smithi089.stdout:007 2023-07-06T02:25:29.441 INFO:teuthology.orchestra.run.smithi089.stdout:007.out 2023-07-06T02:25:29.441 INFO:teuthology.orchestra.run.smithi089.stdout:008 2023-07-06T02:25:29.441 INFO:teuthology.orchestra.run.smithi089.stdout:008.out 2023-07-06T02:25:29.441 INFO:teuthology.orchestra.run.smithi089.stdout:009 2023-07-06T02:25:29.441 INFO:teuthology.orchestra.run.smithi089.stdout:009.out 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:010 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:010.out 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:011 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:011.out 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:012 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:012.out 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:013 2023-07-06T02:25:29.442 INFO:teuthology.orchestra.run.smithi089.stdout:013.out 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:014 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:014.out 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:015 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:015.out 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:016 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:016.out 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:017 2023-07-06T02:25:29.443 INFO:teuthology.orchestra.run.smithi089.stdout:017.out 2023-07-06T02:25:29.444 INFO:teuthology.orchestra.run.smithi089.stdout:018 2023-07-06T02:25:29.444 INFO:teuthology.orchestra.run.smithi089.stdout:018.out 2023-07-06T02:25:29.444 INFO:teuthology.orchestra.run.smithi089.stdout:019 2023-07-06T02:25:29.444 INFO:teuthology.orchestra.run.smithi089.stdout:019.out 2023-07-06T02:25:29.444 INFO:teuthology.orchestra.run.smithi089.stdout:020 2023-07-06T02:25:29.444 INFO:teuthology.orchestra.run.smithi089.stdout:020.out 2023-07-06T02:25:29.444 INFO:teuthology.orchestra.run.smithi089.stdout:021 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:021.out 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:022 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:022.out 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:023 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:023.out 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:024 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:024.out 2023-07-06T02:25:29.445 INFO:teuthology.orchestra.run.smithi089.stdout:025 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:025.out 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:026 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:026.out 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:027 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:027.out 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:028 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:028.out 2023-07-06T02:25:29.446 INFO:teuthology.orchestra.run.smithi089.stdout:029 2023-07-06T02:25:29.447 INFO:teuthology.orchestra.run.smithi089.stdout:029.out 2023-07-06T02:25:29.447 INFO:teuthology.orchestra.run.smithi089.stdout:030 2023-07-06T02:25:29.447 INFO:teuthology.orchestra.run.smithi089.stdout:030.out 2023-07-06T02:25:29.447 INFO:teuthology.orchestra.run.smithi089.stdout:031 2023-07-06T02:25:29.447 INFO:teuthology.orchestra.run.smithi089.stdout:031.out 2023-07-06T02:25:29.447 INFO:teuthology.orchestra.run.smithi089.stdout:032 2023-07-06T02:25:29.447 INFO:teuthology.orchestra.run.smithi089.stdout:032.out 2023-07-06T02:25:29.448 INFO:teuthology.orchestra.run.smithi089.stdout:033 2023-07-06T02:25:29.448 INFO:teuthology.orchestra.run.smithi089.stdout:033.out 2023-07-06T02:25:29.448 INFO:teuthology.orchestra.run.smithi089.stdout:034 2023-07-06T02:25:29.448 INFO:teuthology.orchestra.run.smithi089.stdout:034.out 2023-07-06T02:25:29.448 INFO:teuthology.orchestra.run.smithi089.stdout:035 2023-07-06T02:25:29.448 INFO:teuthology.orchestra.run.smithi089.stdout:035.cfg 2023-07-06T02:25:29.449 INFO:teuthology.orchestra.run.smithi089.stdout:035.out.default 2023-07-06T02:25:29.449 INFO:teuthology.orchestra.run.smithi089.stdout:035.out.nfs 2023-07-06T02:25:29.449 INFO:teuthology.orchestra.run.smithi089.stdout:036 2023-07-06T02:25:29.449 INFO:teuthology.orchestra.run.smithi089.stdout:036.out 2023-07-06T02:25:29.449 INFO:teuthology.orchestra.run.smithi089.stdout:037 2023-07-06T02:25:29.449 INFO:teuthology.orchestra.run.smithi089.stdout:037.out 2023-07-06T02:25:29.449 INFO:teuthology.orchestra.run.smithi089.stdout:038 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:038.out 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:039 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:039.out 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:040 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:040.out 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:041 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:041.out 2023-07-06T02:25:29.450 INFO:teuthology.orchestra.run.smithi089.stdout:042 2023-07-06T02:25:29.451 INFO:teuthology.orchestra.run.smithi089.stdout:042.out 2023-07-06T02:25:29.451 INFO:teuthology.orchestra.run.smithi089.stdout:043 2023-07-06T02:25:29.451 INFO:teuthology.orchestra.run.smithi089.stdout:043.out 2023-07-06T02:25:29.451 INFO:teuthology.orchestra.run.smithi089.stdout:044 2023-07-06T02:25:29.451 INFO:teuthology.orchestra.run.smithi089.stdout:044.out 2023-07-06T02:25:29.451 INFO:teuthology.orchestra.run.smithi089.stdout:045 2023-07-06T02:25:29.451 INFO:teuthology.orchestra.run.smithi089.stdout:045.out 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:046 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:046.out 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:047 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:047.out 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:048 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:048.out 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:049 2023-07-06T02:25:29.452 INFO:teuthology.orchestra.run.smithi089.stdout:049.out 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:050 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:050.cfg 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:050.out.default 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:050.out.nojournal 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:050.out.xfsquota 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:051 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:051.out 2023-07-06T02:25:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:052 2023-07-06T02:25:29.454 INFO:teuthology.orchestra.run.smithi089.stdout:052.out 2023-07-06T02:25:29.454 INFO:teuthology.orchestra.run.smithi089.stdout:053 2023-07-06T02:25:29.454 INFO:teuthology.orchestra.run.smithi089.stdout:053.out 2023-07-06T02:25:29.454 INFO:teuthology.orchestra.run.smithi089.stdout:054 2023-07-06T02:25:29.454 INFO:teuthology.orchestra.run.smithi089.stdout:054.out 2023-07-06T02:25:29.454 INFO:teuthology.orchestra.run.smithi089.stdout:055 2023-07-06T02:25:29.454 INFO:teuthology.orchestra.run.smithi089.stdout:055.out 2023-07-06T02:25:29.455 INFO:teuthology.orchestra.run.smithi089.stdout:056 2023-07-06T02:25:29.455 INFO:teuthology.orchestra.run.smithi089.stdout:056.out 2023-07-06T02:25:29.455 INFO:teuthology.orchestra.run.smithi089.stdout:057 2023-07-06T02:25:29.455 INFO:teuthology.orchestra.run.smithi089.stdout:057.out 2023-07-06T02:25:29.455 INFO:teuthology.orchestra.run.smithi089.stdout:058 2023-07-06T02:25:29.455 INFO:teuthology.orchestra.run.smithi089.stdout:058.out 2023-07-06T02:25:29.455 INFO:teuthology.orchestra.run.smithi089.stdout:059 2023-07-06T02:25:29.456 INFO:teuthology.orchestra.run.smithi089.stdout:059.out 2023-07-06T02:25:29.456 INFO:teuthology.orchestra.run.smithi089.stdout:060 2023-07-06T02:25:29.456 INFO:teuthology.orchestra.run.smithi089.stdout:060.out 2023-07-06T02:25:29.456 INFO:teuthology.orchestra.run.smithi089.stdout:061 2023-07-06T02:25:29.456 INFO:teuthology.orchestra.run.smithi089.stdout:061.out 2023-07-06T02:25:29.456 INFO:teuthology.orchestra.run.smithi089.stdout:062 2023-07-06T02:25:29.456 INFO:teuthology.orchestra.run.smithi089.stdout:062.out 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:063 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:063.out 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:064 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:064.out 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:065 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:065.out 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:066 2023-07-06T02:25:29.457 INFO:teuthology.orchestra.run.smithi089.stdout:066.out 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:067 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:067.out 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:068 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:068.out 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:069 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:069.out 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:070 2023-07-06T02:25:29.458 INFO:teuthology.orchestra.run.smithi089.stdout:070.out 2023-07-06T02:25:29.459 INFO:teuthology.orchestra.run.smithi089.stdout:071 2023-07-06T02:25:29.459 INFO:teuthology.orchestra.run.smithi089.stdout:071.out 2023-07-06T02:25:29.459 INFO:teuthology.orchestra.run.smithi089.stdout:072 2023-07-06T02:25:29.459 INFO:teuthology.orchestra.run.smithi089.stdout:072.out 2023-07-06T02:25:29.459 INFO:teuthology.orchestra.run.smithi089.stdout:073 2023-07-06T02:25:29.459 INFO:teuthology.orchestra.run.smithi089.stdout:073.out 2023-07-06T02:25:29.459 INFO:teuthology.orchestra.run.smithi089.stdout:074 2023-07-06T02:25:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:074.out 2023-07-06T02:25:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:075 2023-07-06T02:25:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:075.out 2023-07-06T02:25:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:076 2023-07-06T02:25:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:076.out 2023-07-06T02:25:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:077 2023-07-06T02:25:29.460 INFO:teuthology.orchestra.run.smithi089.stdout:077.out 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:078 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:078.out 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:079 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:079.out 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:080 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:080.out 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:081 2023-07-06T02:25:29.461 INFO:teuthology.orchestra.run.smithi089.stdout:081.out 2023-07-06T02:25:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:082 2023-07-06T02:25:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:082.out 2023-07-06T02:25:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:083 2023-07-06T02:25:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:083.out 2023-07-06T02:25:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:084 2023-07-06T02:25:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:084.out 2023-07-06T02:25:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:085 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:085.out 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:086 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:086.out 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:087 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:087.out 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:088 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:088.out 2023-07-06T02:25:29.463 INFO:teuthology.orchestra.run.smithi089.stdout:089 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:089.out 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:090 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:090.out 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:091 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:091.out 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:092 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:092.out 2023-07-06T02:25:29.464 INFO:teuthology.orchestra.run.smithi089.stdout:093 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:093.out 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:094 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:094.out 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:095 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:095.out 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:096 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:096.out 2023-07-06T02:25:29.465 INFO:teuthology.orchestra.run.smithi089.stdout:097 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:097.out 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:098 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:098.out 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:099 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:099.out 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:100 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:100.out 2023-07-06T02:25:29.466 INFO:teuthology.orchestra.run.smithi089.stdout:101 2023-07-06T02:25:29.467 INFO:teuthology.orchestra.run.smithi089.stdout:101.out 2023-07-06T02:25:29.467 INFO:teuthology.orchestra.run.smithi089.stdout:102 2023-07-06T02:25:29.467 INFO:teuthology.orchestra.run.smithi089.stdout:102.out 2023-07-06T02:25:29.467 INFO:teuthology.orchestra.run.smithi089.stdout:103 2023-07-06T02:25:29.467 INFO:teuthology.orchestra.run.smithi089.stdout:103.out 2023-07-06T02:25:29.467 INFO:teuthology.orchestra.run.smithi089.stdout:104 2023-07-06T02:25:29.467 INFO:teuthology.orchestra.run.smithi089.stdout:104.out 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:105 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:105.out 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:106 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:106.out 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:107 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:107.out 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:108 2023-07-06T02:25:29.468 INFO:teuthology.orchestra.run.smithi089.stdout:108.out 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:109 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:109.out 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:110 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:110.out 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:111 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:111.out 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:112 2023-07-06T02:25:29.469 INFO:teuthology.orchestra.run.smithi089.stdout:112.out 2023-07-06T02:25:29.470 INFO:teuthology.orchestra.run.smithi089.stdout:113 2023-07-06T02:25:29.470 INFO:teuthology.orchestra.run.smithi089.stdout:113.out 2023-07-06T02:25:29.470 INFO:teuthology.orchestra.run.smithi089.stdout:114 2023-07-06T02:25:29.470 INFO:teuthology.orchestra.run.smithi089.stdout:114.out 2023-07-06T02:25:29.470 INFO:teuthology.orchestra.run.smithi089.stdout:115 2023-07-06T02:25:29.470 INFO:teuthology.orchestra.run.smithi089.stdout:115.out 2023-07-06T02:25:29.470 INFO:teuthology.orchestra.run.smithi089.stdout:116 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:116.out 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:117 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:117.out 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:118 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:118.out 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:119 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:119.out 2023-07-06T02:25:29.471 INFO:teuthology.orchestra.run.smithi089.stdout:120 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:120.out 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:121 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:121.out 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:122 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:122.out 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:123 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:123.out 2023-07-06T02:25:29.472 INFO:teuthology.orchestra.run.smithi089.stdout:124 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:124.out 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:125 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:125.out 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:126 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:126.out 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:127 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:127.out 2023-07-06T02:25:29.473 INFO:teuthology.orchestra.run.smithi089.stdout:128 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:128.out 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:129 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:129.out 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:130 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:130.out 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:131 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:131.out 2023-07-06T02:25:29.474 INFO:teuthology.orchestra.run.smithi089.stdout:132 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:132.out 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:133 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:133.out 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:134 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:134.out 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:135 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:135.out 2023-07-06T02:25:29.475 INFO:teuthology.orchestra.run.smithi089.stdout:136 2023-07-06T02:25:29.476 INFO:teuthology.orchestra.run.smithi089.stdout:136.out 2023-07-06T02:25:29.476 INFO:teuthology.orchestra.run.smithi089.stdout:137 2023-07-06T02:25:29.476 INFO:teuthology.orchestra.run.smithi089.stdout:137.out 2023-07-06T02:25:29.476 INFO:teuthology.orchestra.run.smithi089.stdout:138 2023-07-06T02:25:29.476 INFO:teuthology.orchestra.run.smithi089.stdout:138.out 2023-07-06T02:25:29.476 INFO:teuthology.orchestra.run.smithi089.stdout:139 2023-07-06T02:25:29.476 INFO:teuthology.orchestra.run.smithi089.stdout:139.out 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:140 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:140.out 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:141 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:141.out 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:142 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:142.out 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:143 2023-07-06T02:25:29.477 INFO:teuthology.orchestra.run.smithi089.stdout:143.out 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:144 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:144.out 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:145 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:145.out 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:146 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:146.out 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:147 2023-07-06T02:25:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:147.out 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:148 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:148.out 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:149 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:149.out 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:150 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:150.out 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:151 2023-07-06T02:25:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:151.out 2023-07-06T02:25:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:152 2023-07-06T02:25:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:152.out 2023-07-06T02:25:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:153 2023-07-06T02:25:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:153.out 2023-07-06T02:25:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:154 2023-07-06T02:25:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:154.out 2023-07-06T02:25:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:155 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:155.out 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:156 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:156.out 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:157 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:157.out 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:158 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:158.out 2023-07-06T02:25:29.481 INFO:teuthology.orchestra.run.smithi089.stdout:159 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:159.out 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:160 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:160.out 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:161 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:161.out 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:162 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:162.out 2023-07-06T02:25:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:163 2023-07-06T02:25:29.483 INFO:teuthology.orchestra.run.smithi089.stdout:163.out 2023-07-06T02:25:29.483 INFO:teuthology.orchestra.run.smithi089.stdout:164 2023-07-06T02:25:29.483 INFO:teuthology.orchestra.run.smithi089.stdout:164.out 2023-07-06T02:25:29.483 INFO:teuthology.orchestra.run.smithi089.stdout:165 2023-07-06T02:25:29.483 INFO:teuthology.orchestra.run.smithi089.stdout:165.out 2023-07-06T02:25:29.483 INFO:teuthology.orchestra.run.smithi089.stdout:166 2023-07-06T02:25:29.483 INFO:teuthology.orchestra.run.smithi089.stdout:166.out 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:167 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:167.out 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:168 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:168.out 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:169 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:169.out 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:170 2023-07-06T02:25:29.484 INFO:teuthology.orchestra.run.smithi089.stdout:170.out 2023-07-06T02:25:29.485 INFO:teuthology.orchestra.run.smithi089.stdout:171 2023-07-06T02:25:29.485 INFO:teuthology.orchestra.run.smithi089.stdout:171.out 2023-07-06T02:25:29.485 INFO:teuthology.orchestra.run.smithi089.stdout:172 2023-07-06T02:25:29.485 INFO:teuthology.orchestra.run.smithi089.stdout:172.out 2023-07-06T02:25:29.485 INFO:teuthology.orchestra.run.smithi089.stdout:173 2023-07-06T02:25:29.485 INFO:teuthology.orchestra.run.smithi089.stdout:173.out 2023-07-06T02:25:29.485 INFO:teuthology.orchestra.run.smithi089.stdout:174 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:174.out 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:175 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:175.out 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:176 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:176.out 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:177 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:177.out 2023-07-06T02:25:29.486 INFO:teuthology.orchestra.run.smithi089.stdout:178 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:178.out 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:179 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:179.out 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:180 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:180.out 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:181 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:181.out 2023-07-06T02:25:29.487 INFO:teuthology.orchestra.run.smithi089.stdout:182 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:182.out 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:183 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:183.out 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:184 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:184.out 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:185 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:185.out 2023-07-06T02:25:29.488 INFO:teuthology.orchestra.run.smithi089.stdout:186 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:186.out 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:187 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:187.out 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:188 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:188.out 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:189 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:189.out 2023-07-06T02:25:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:190 2023-07-06T02:25:29.490 INFO:teuthology.orchestra.run.smithi089.stdout:190.out 2023-07-06T02:25:29.490 INFO:teuthology.orchestra.run.smithi089.stdout:191 2023-07-06T02:25:29.490 INFO:teuthology.orchestra.run.smithi089.stdout:191.out 2023-07-06T02:25:29.490 INFO:teuthology.orchestra.run.smithi089.stdout:192 2023-07-06T02:25:29.490 INFO:teuthology.orchestra.run.smithi089.stdout:192.out 2023-07-06T02:25:29.490 INFO:teuthology.orchestra.run.smithi089.stdout:193 2023-07-06T02:25:29.490 INFO:teuthology.orchestra.run.smithi089.stdout:193.out 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:194 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:194.out 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:195 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:195.out 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:196 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:196.out 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:197 2023-07-06T02:25:29.491 INFO:teuthology.orchestra.run.smithi089.stdout:197.out 2023-07-06T02:25:29.492 INFO:teuthology.orchestra.run.smithi089.stdout:198 2023-07-06T02:25:29.492 INFO:teuthology.orchestra.run.smithi089.stdout:198.out 2023-07-06T02:25:29.492 INFO:teuthology.orchestra.run.smithi089.stdout:199 2023-07-06T02:25:29.492 INFO:teuthology.orchestra.run.smithi089.stdout:199.out 2023-07-06T02:25:29.492 INFO:teuthology.orchestra.run.smithi089.stdout:200 2023-07-06T02:25:29.492 INFO:teuthology.orchestra.run.smithi089.stdout:200.out 2023-07-06T02:25:29.492 INFO:teuthology.orchestra.run.smithi089.stdout:201 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:201.out 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:202 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:202.out 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:203 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:203.out 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:204 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:204.out 2023-07-06T02:25:29.493 INFO:teuthology.orchestra.run.smithi089.stdout:205 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:205.out 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:206 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:206.out 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:207 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:207.out 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:208 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:208.out 2023-07-06T02:25:29.494 INFO:teuthology.orchestra.run.smithi089.stdout:209 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:209.out 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:210 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:210.out 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:211 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:211.out 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:212 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:212.out 2023-07-06T02:25:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:213 2023-07-06T02:25:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:213.out 2023-07-06T02:25:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:214 2023-07-06T02:25:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:214.out 2023-07-06T02:25:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:215 2023-07-06T02:25:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:215.out 2023-07-06T02:25:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:216 2023-07-06T02:25:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:216.out 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:217 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:217.out 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:218 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:218.out 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:219 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:219.out 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:220 2023-07-06T02:25:29.497 INFO:teuthology.orchestra.run.smithi089.stdout:220.out 2023-07-06T02:25:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:221 2023-07-06T02:25:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:221.out 2023-07-06T02:25:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:222 2023-07-06T02:25:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:222.out 2023-07-06T02:25:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:223 2023-07-06T02:25:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:223.out 2023-07-06T02:25:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:224 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:224.out 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:225 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:225.out 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:226 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:226.out 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:227 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:227.out 2023-07-06T02:25:29.499 INFO:teuthology.orchestra.run.smithi089.stdout:228 2023-07-06T02:25:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:228.out 2023-07-06T02:25:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:229 2023-07-06T02:25:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:229.out 2023-07-06T02:25:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:230 2023-07-06T02:25:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:230.out 2023-07-06T02:25:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:231 2023-07-06T02:25:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:231.out 2023-07-06T02:25:29.501 INFO:teuthology.orchestra.run.smithi089.stdout:232 2023-07-06T02:25:29.501 INFO:teuthology.orchestra.run.smithi089.stdout:232.out 2023-07-06T02:25:29.501 INFO:teuthology.orchestra.run.smithi089.stdout:233 2023-07-06T02:25:29.501 INFO:teuthology.orchestra.run.smithi089.stdout:233.out 2023-07-06T02:25:29.501 INFO:teuthology.orchestra.run.smithi089.stdout:234 2023-07-06T02:25:29.501 INFO:teuthology.orchestra.run.smithi089.stdout:234.out 2023-07-06T02:25:29.501 INFO:teuthology.orchestra.run.smithi089.stdout:235 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:235.out 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:236 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:236.out 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:237 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:237.out 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:238 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:238.out 2023-07-06T02:25:29.502 INFO:teuthology.orchestra.run.smithi089.stdout:239 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:239.out 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:240 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:240.out 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:241 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:241.out 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:242 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:242.out 2023-07-06T02:25:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:243 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:243.out 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:244 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:244.out 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:245 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:245.out 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:246 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:246.out 2023-07-06T02:25:29.504 INFO:teuthology.orchestra.run.smithi089.stdout:247 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:247.out 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:248 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:248.out 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:249 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:249.out 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:250 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:250.out 2023-07-06T02:25:29.505 INFO:teuthology.orchestra.run.smithi089.stdout:251 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:251.out 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:252 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:252.out 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:253 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:253.out 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:254 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:254.out 2023-07-06T02:25:29.506 INFO:teuthology.orchestra.run.smithi089.stdout:255 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:255.out 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:256 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:256.out 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:257 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:257.out 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:258 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:258.out 2023-07-06T02:25:29.507 INFO:teuthology.orchestra.run.smithi089.stdout:259 2023-07-06T02:25:29.508 INFO:teuthology.orchestra.run.smithi089.stdout:259.out 2023-07-06T02:25:29.508 INFO:teuthology.orchestra.run.smithi089.stdout:260 2023-07-06T02:25:29.508 INFO:teuthology.orchestra.run.smithi089.stdout:260.out 2023-07-06T02:25:29.508 INFO:teuthology.orchestra.run.smithi089.stdout:261 2023-07-06T02:25:29.508 INFO:teuthology.orchestra.run.smithi089.stdout:261.out 2023-07-06T02:25:29.508 INFO:teuthology.orchestra.run.smithi089.stdout:262 2023-07-06T02:25:29.508 INFO:teuthology.orchestra.run.smithi089.stdout:262.out 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:263 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:263.out 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:264 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:264.out 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:265 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:265.out 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:266 2023-07-06T02:25:29.509 INFO:teuthology.orchestra.run.smithi089.stdout:266.out 2023-07-06T02:25:29.510 INFO:teuthology.orchestra.run.smithi089.stdout:267 2023-07-06T02:25:29.510 INFO:teuthology.orchestra.run.smithi089.stdout:267.out 2023-07-06T02:25:29.510 INFO:teuthology.orchestra.run.smithi089.stdout:268 2023-07-06T02:25:29.510 INFO:teuthology.orchestra.run.smithi089.stdout:268.out 2023-07-06T02:25:29.510 INFO:teuthology.orchestra.run.smithi089.stdout:269 2023-07-06T02:25:29.510 INFO:teuthology.orchestra.run.smithi089.stdout:269.out 2023-07-06T02:25:29.510 INFO:teuthology.orchestra.run.smithi089.stdout:270 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:270.out 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:271 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:271.out 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:272 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:272.out 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:273 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:273.out 2023-07-06T02:25:29.511 INFO:teuthology.orchestra.run.smithi089.stdout:274 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:274.out 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:275 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:275.out 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:276 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:276.out 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:277 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:277.out 2023-07-06T02:25:29.512 INFO:teuthology.orchestra.run.smithi089.stdout:278 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:278.out 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:279 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:279.out 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:280 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:280.out 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:281 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:281.out 2023-07-06T02:25:29.513 INFO:teuthology.orchestra.run.smithi089.stdout:282 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:282.out 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:283 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:283.out 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:284 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:284.out 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:285 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:285.out 2023-07-06T02:25:29.514 INFO:teuthology.orchestra.run.smithi089.stdout:286 2023-07-06T02:25:29.515 INFO:teuthology.orchestra.run.smithi089.stdout:286.out 2023-07-06T02:25:29.515 INFO:teuthology.orchestra.run.smithi089.stdout:287 2023-07-06T02:25:29.515 INFO:teuthology.orchestra.run.smithi089.stdout:287.out 2023-07-06T02:25:29.515 INFO:teuthology.orchestra.run.smithi089.stdout:288 2023-07-06T02:25:29.515 INFO:teuthology.orchestra.run.smithi089.stdout:288.out 2023-07-06T02:25:29.515 INFO:teuthology.orchestra.run.smithi089.stdout:289 2023-07-06T02:25:29.515 INFO:teuthology.orchestra.run.smithi089.stdout:289.out 2023-07-06T02:25:29.516 INFO:teuthology.orchestra.run.smithi089.stdout:290 2023-07-06T02:25:29.516 INFO:teuthology.orchestra.run.smithi089.stdout:290.out 2023-07-06T02:25:29.516 INFO:teuthology.orchestra.run.smithi089.stdout:291 2023-07-06T02:25:29.516 INFO:teuthology.orchestra.run.smithi089.stdout:291.out 2023-07-06T02:25:29.516 INFO:teuthology.orchestra.run.smithi089.stdout:292 2023-07-06T02:25:29.516 INFO:teuthology.orchestra.run.smithi089.stdout:292.out 2023-07-06T02:25:29.516 INFO:teuthology.orchestra.run.smithi089.stdout:293 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:293.out 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:294 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:294.out 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:295 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:295.out 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:296 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:296.out 2023-07-06T02:25:29.517 INFO:teuthology.orchestra.run.smithi089.stdout:297 2023-07-06T02:25:29.518 INFO:teuthology.orchestra.run.smithi089.stdout:297.out 2023-07-06T02:25:29.518 INFO:teuthology.orchestra.run.smithi089.stdout:298 2023-07-06T02:25:29.518 INFO:teuthology.orchestra.run.smithi089.stdout:298.out 2023-07-06T02:25:29.518 INFO:teuthology.orchestra.run.smithi089.stdout:299 2023-07-06T02:25:29.518 INFO:teuthology.orchestra.run.smithi089.stdout:299.out 2023-07-06T02:25:29.518 INFO:teuthology.orchestra.run.smithi089.stdout:300 2023-07-06T02:25:29.518 INFO:teuthology.orchestra.run.smithi089.stdout:300.out 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:301 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:301.out 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:302 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:302.out 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:303 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:303.out 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:304 2023-07-06T02:25:29.519 INFO:teuthology.orchestra.run.smithi089.stdout:304.out 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:305 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:305.out 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:306 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:306.out 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:307 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:307.out 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:308 2023-07-06T02:25:29.520 INFO:teuthology.orchestra.run.smithi089.stdout:308.out 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:309 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:309.out 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:310 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:310.out 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:311 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:311.out 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:312 2023-07-06T02:25:29.521 INFO:teuthology.orchestra.run.smithi089.stdout:312.out 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:313 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:313.out 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:314 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:314.out 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:315 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:315.out 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:316 2023-07-06T02:25:29.522 INFO:teuthology.orchestra.run.smithi089.stdout:316.out 2023-07-06T02:25:29.523 INFO:teuthology.orchestra.run.smithi089.stdout:317 2023-07-06T02:25:29.523 INFO:teuthology.orchestra.run.smithi089.stdout:317.out 2023-07-06T02:25:29.523 INFO:teuthology.orchestra.run.smithi089.stdout:318 2023-07-06T02:25:29.523 INFO:teuthology.orchestra.run.smithi089.stdout:318.out 2023-07-06T02:25:29.523 INFO:teuthology.orchestra.run.smithi089.stdout:319 2023-07-06T02:25:29.523 INFO:teuthology.orchestra.run.smithi089.stdout:319.out 2023-07-06T02:25:29.523 INFO:teuthology.orchestra.run.smithi089.stdout:320 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:320.out 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:321 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:321.out 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:322 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:322.out 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:323 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:323.out 2023-07-06T02:25:29.524 INFO:teuthology.orchestra.run.smithi089.stdout:324 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:324.out 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:325 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:325.out 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:326 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:326.out 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:327 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:327.out 2023-07-06T02:25:29.525 INFO:teuthology.orchestra.run.smithi089.stdout:328 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:328.out 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:329 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:329.out 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:330 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:330.out 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:331 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:331.out 2023-07-06T02:25:29.526 INFO:teuthology.orchestra.run.smithi089.stdout:332 2023-07-06T02:25:29.527 INFO:teuthology.orchestra.run.smithi089.stdout:332.out 2023-07-06T02:25:29.527 INFO:teuthology.orchestra.run.smithi089.stdout:333 2023-07-06T02:25:29.527 INFO:teuthology.orchestra.run.smithi089.stdout:333.out 2023-07-06T02:25:29.527 INFO:teuthology.orchestra.run.smithi089.stdout:334 2023-07-06T02:25:29.527 INFO:teuthology.orchestra.run.smithi089.stdout:334.out 2023-07-06T02:25:29.527 INFO:teuthology.orchestra.run.smithi089.stdout:335 2023-07-06T02:25:29.527 INFO:teuthology.orchestra.run.smithi089.stdout:335.out 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:336 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:336.out 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:337 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:337.out 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:338 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:338.out 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:339 2023-07-06T02:25:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:339.out 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:340 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:340.out 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:341 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:341.out 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:342 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:342.out 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:343 2023-07-06T02:25:29.529 INFO:teuthology.orchestra.run.smithi089.stdout:343.out 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:344 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:344.out 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:345 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:345.out 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:346 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:346.out 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:347 2023-07-06T02:25:29.530 INFO:teuthology.orchestra.run.smithi089.stdout:347.out 2023-07-06T02:25:29.531 INFO:teuthology.orchestra.run.smithi089.stdout:348 2023-07-06T02:25:29.531 INFO:teuthology.orchestra.run.smithi089.stdout:348.out 2023-07-06T02:25:29.531 INFO:teuthology.orchestra.run.smithi089.stdout:349 2023-07-06T02:25:29.531 INFO:teuthology.orchestra.run.smithi089.stdout:349.out 2023-07-06T02:25:29.531 INFO:teuthology.orchestra.run.smithi089.stdout:350 2023-07-06T02:25:29.531 INFO:teuthology.orchestra.run.smithi089.stdout:350.out 2023-07-06T02:25:29.531 INFO:teuthology.orchestra.run.smithi089.stdout:351 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:351.out 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:352 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:352.out 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:353 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:353.out 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:354 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:354.out 2023-07-06T02:25:29.532 INFO:teuthology.orchestra.run.smithi089.stdout:355 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:355.out 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:356 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:356.out 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:357 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:357.out 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:358 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:358.out 2023-07-06T02:25:29.533 INFO:teuthology.orchestra.run.smithi089.stdout:359 2023-07-06T02:25:29.534 INFO:teuthology.orchestra.run.smithi089.stdout:359.out 2023-07-06T02:25:29.534 INFO:teuthology.orchestra.run.smithi089.stdout:360 2023-07-06T02:25:29.534 INFO:teuthology.orchestra.run.smithi089.stdout:360.out 2023-07-06T02:25:29.534 INFO:teuthology.orchestra.run.smithi089.stdout:361 2023-07-06T02:25:29.534 INFO:teuthology.orchestra.run.smithi089.stdout:361.out 2023-07-06T02:25:29.534 INFO:teuthology.orchestra.run.smithi089.stdout:362 2023-07-06T02:25:29.534 INFO:teuthology.orchestra.run.smithi089.stdout:362.out 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:363 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:363.out 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:364 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:364.out 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:365 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:365.out 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:366 2023-07-06T02:25:29.535 INFO:teuthology.orchestra.run.smithi089.stdout:366.out 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:367 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:367.out 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:368 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:368.out 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:369 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:369.out 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:370 2023-07-06T02:25:29.536 INFO:teuthology.orchestra.run.smithi089.stdout:370.out 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:371 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:371.out 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:372 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:372.out 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:373 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:373.out 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:374 2023-07-06T02:25:29.537 INFO:teuthology.orchestra.run.smithi089.stdout:374.out 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:375 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:375.out 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:376 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:376.out 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:377 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:377.out 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:378 2023-07-06T02:25:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:378.out 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:379 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:379.out 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:380 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:380.out 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:381 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:381.out 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:382 2023-07-06T02:25:29.539 INFO:teuthology.orchestra.run.smithi089.stdout:382.out 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:383 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:383.out 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:384 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:384.out 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:385 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:385.out 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:386 2023-07-06T02:25:29.540 INFO:teuthology.orchestra.run.smithi089.stdout:386.out 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:387 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:387.out 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:388 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:388.out 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:389 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:389.out 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:390 2023-07-06T02:25:29.541 INFO:teuthology.orchestra.run.smithi089.stdout:390.out 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:391 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:391.out 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:392 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:392.out 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:393 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:393.out 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:394 2023-07-06T02:25:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:394.out 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:395 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:395.out 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:396 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:396.out 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:397 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:397.out 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:398 2023-07-06T02:25:29.543 INFO:teuthology.orchestra.run.smithi089.stdout:398.out 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:399 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:399.out 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:400 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:400.out 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:401 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:401.out 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:402 2023-07-06T02:25:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:402.out 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:403 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:403.out 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:404 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:404.out 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:405 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:405.out 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:406 2023-07-06T02:25:29.545 INFO:teuthology.orchestra.run.smithi089.stdout:406.out 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:407 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:407.out 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:408 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:408.out 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:409 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:409.out 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:410 2023-07-06T02:25:29.546 INFO:teuthology.orchestra.run.smithi089.stdout:410.out 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:411 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:411.out 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:412 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:412.out 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:413 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:413.out 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:414 2023-07-06T02:25:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:414.out 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:415 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:415.out 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:416 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:416.out 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:417 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:417.out 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:418 2023-07-06T02:25:29.548 INFO:teuthology.orchestra.run.smithi089.stdout:418.out 2023-07-06T02:25:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:419 2023-07-06T02:25:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:419.out 2023-07-06T02:25:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:420 2023-07-06T02:25:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:420.out 2023-07-06T02:25:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:421 2023-07-06T02:25:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:421.out 2023-07-06T02:25:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:422 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:422.out 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:423 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:423.out 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:424 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:424.out 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:425 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:425.out 2023-07-06T02:25:29.550 INFO:teuthology.orchestra.run.smithi089.stdout:426 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:426.out 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:427 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:427.out 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:428 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:428.out 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:429 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:429.out 2023-07-06T02:25:29.551 INFO:teuthology.orchestra.run.smithi089.stdout:430 2023-07-06T02:25:29.552 INFO:teuthology.orchestra.run.smithi089.stdout:430.out 2023-07-06T02:25:29.552 INFO:teuthology.orchestra.run.smithi089.stdout:431 2023-07-06T02:25:29.552 INFO:teuthology.orchestra.run.smithi089.stdout:431.out 2023-07-06T02:25:29.552 INFO:teuthology.orchestra.run.smithi089.stdout:432 2023-07-06T02:25:29.552 INFO:teuthology.orchestra.run.smithi089.stdout:432.out 2023-07-06T02:25:29.552 INFO:teuthology.orchestra.run.smithi089.stdout:433 2023-07-06T02:25:29.552 INFO:teuthology.orchestra.run.smithi089.stdout:433.out 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:434 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:434.out 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:435 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:435.out 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:436 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:436.out 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:437 2023-07-06T02:25:29.553 INFO:teuthology.orchestra.run.smithi089.stdout:437.out 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:438 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:438.out 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:439 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:439.out 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:440 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:440.out 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:441 2023-07-06T02:25:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:441.out 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:442 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:442.out 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:443 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:443.out 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:444 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:444.out 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:445 2023-07-06T02:25:29.555 INFO:teuthology.orchestra.run.smithi089.stdout:445.out 2023-07-06T02:25:29.556 INFO:teuthology.orchestra.run.smithi089.stdout:446 2023-07-06T02:25:29.556 INFO:teuthology.orchestra.run.smithi089.stdout:446.out 2023-07-06T02:25:29.556 INFO:teuthology.orchestra.run.smithi089.stdout:447 2023-07-06T02:25:29.556 INFO:teuthology.orchestra.run.smithi089.stdout:447.out 2023-07-06T02:25:29.556 INFO:teuthology.orchestra.run.smithi089.stdout:448 2023-07-06T02:25:29.556 INFO:teuthology.orchestra.run.smithi089.stdout:448.out 2023-07-06T02:25:29.556 INFO:teuthology.orchestra.run.smithi089.stdout:449 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:449.out 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:450 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:450.out 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:451 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:451.out 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:452 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:452.out 2023-07-06T02:25:29.557 INFO:teuthology.orchestra.run.smithi089.stdout:453 2023-07-06T02:25:29.558 INFO:teuthology.orchestra.run.smithi089.stdout:453.out 2023-07-06T02:25:29.558 INFO:teuthology.orchestra.run.smithi089.stdout:454 2023-07-06T02:25:29.558 INFO:teuthology.orchestra.run.smithi089.stdout:454.out 2023-07-06T02:25:29.558 INFO:teuthology.orchestra.run.smithi089.stdout:455 2023-07-06T02:25:29.558 INFO:teuthology.orchestra.run.smithi089.stdout:455.out 2023-07-06T02:25:29.558 INFO:teuthology.orchestra.run.smithi089.stdout:456 2023-07-06T02:25:29.558 INFO:teuthology.orchestra.run.smithi089.stdout:456.out 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:457 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:457.out 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:458 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:458.out 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:459 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:459.out 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:460 2023-07-06T02:25:29.559 INFO:teuthology.orchestra.run.smithi089.stdout:460.out 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:461 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:461.out 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:462 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:462.out 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:463 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:463.out 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:464 2023-07-06T02:25:29.560 INFO:teuthology.orchestra.run.smithi089.stdout:464.out 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:465 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:465.out 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:466 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:466.out 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:467 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:467.out 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:468 2023-07-06T02:25:29.561 INFO:teuthology.orchestra.run.smithi089.stdout:468.out 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:469 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:469.out 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:470 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:470.out 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:471 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:471.out 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:472 2023-07-06T02:25:29.562 INFO:teuthology.orchestra.run.smithi089.stdout:472.out 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:473 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:473.out 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:474 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:474.out 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:475 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:475.out 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:476 2023-07-06T02:25:29.563 INFO:teuthology.orchestra.run.smithi089.stdout:476.out 2023-07-06T02:25:29.564 INFO:teuthology.orchestra.run.smithi089.stdout:477 2023-07-06T02:25:29.564 INFO:teuthology.orchestra.run.smithi089.stdout:477.out 2023-07-06T02:25:29.564 INFO:teuthology.orchestra.run.smithi089.stdout:478 2023-07-06T02:25:29.564 INFO:teuthology.orchestra.run.smithi089.stdout:478.out 2023-07-06T02:25:29.564 INFO:teuthology.orchestra.run.smithi089.stdout:479 2023-07-06T02:25:29.564 INFO:teuthology.orchestra.run.smithi089.stdout:479.out 2023-07-06T02:25:29.564 INFO:teuthology.orchestra.run.smithi089.stdout:480 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:480.out 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:481 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:481.out 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:482 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:482.out 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:483 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:483.out 2023-07-06T02:25:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:484 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:484.out 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:485 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:485.out 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:486 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:486.out 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:487 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:487.out 2023-07-06T02:25:29.566 INFO:teuthology.orchestra.run.smithi089.stdout:488 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:488.out 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:489 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:489.out 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:490 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:490.out 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:491 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:491.out 2023-07-06T02:25:29.567 INFO:teuthology.orchestra.run.smithi089.stdout:492 2023-07-06T02:25:29.568 INFO:teuthology.orchestra.run.smithi089.stdout:492.out 2023-07-06T02:25:29.568 INFO:teuthology.orchestra.run.smithi089.stdout:493 2023-07-06T02:25:29.568 INFO:teuthology.orchestra.run.smithi089.stdout:493.out 2023-07-06T02:25:29.568 INFO:teuthology.orchestra.run.smithi089.stdout:494 2023-07-06T02:25:29.568 INFO:teuthology.orchestra.run.smithi089.stdout:494.out 2023-07-06T02:25:29.568 INFO:teuthology.orchestra.run.smithi089.stdout:495 2023-07-06T02:25:29.568 INFO:teuthology.orchestra.run.smithi089.stdout:495.out 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:496 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:496.out 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:497 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:497.out 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:498 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:498.out 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:499 2023-07-06T02:25:29.569 INFO:teuthology.orchestra.run.smithi089.stdout:499.out 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:500 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:500.out 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:501 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:501.out 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:502 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:502.out 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:503 2023-07-06T02:25:29.570 INFO:teuthology.orchestra.run.smithi089.stdout:503.out 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:504 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:504.out 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:505 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:505.out 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:506 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:506.out 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:507 2023-07-06T02:25:29.571 INFO:teuthology.orchestra.run.smithi089.stdout:507.out 2023-07-06T02:25:29.572 INFO:teuthology.orchestra.run.smithi089.stdout:508 2023-07-06T02:25:29.572 INFO:teuthology.orchestra.run.smithi089.stdout:508.out 2023-07-06T02:25:29.572 INFO:teuthology.orchestra.run.smithi089.stdout:509 2023-07-06T02:25:29.572 INFO:teuthology.orchestra.run.smithi089.stdout:509.out 2023-07-06T02:25:29.572 INFO:teuthology.orchestra.run.smithi089.stdout:510 2023-07-06T02:25:29.572 INFO:teuthology.orchestra.run.smithi089.stdout:510.out 2023-07-06T02:25:29.572 INFO:teuthology.orchestra.run.smithi089.stdout:511 2023-07-06T02:25:29.573 INFO:teuthology.orchestra.run.smithi089.stdout:511.out 2023-07-06T02:25:29.573 INFO:teuthology.orchestra.run.smithi089.stdout:512 2023-07-06T02:25:29.573 INFO:teuthology.orchestra.run.smithi089.stdout:512.out 2023-07-06T02:25:29.573 INFO:teuthology.orchestra.run.smithi089.stdout:513 2023-07-06T02:25:29.573 INFO:teuthology.orchestra.run.smithi089.stdout:513.out 2023-07-06T02:25:29.573 INFO:teuthology.orchestra.run.smithi089.stdout:514 2023-07-06T02:25:29.573 INFO:teuthology.orchestra.run.smithi089.stdout:514.out 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:515 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:515.out 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:516 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:516.out 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:517 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:517.out 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:518 2023-07-06T02:25:29.574 INFO:teuthology.orchestra.run.smithi089.stdout:518.out 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:519 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:519.out 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:520 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:520.out 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:521 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:521.out 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:522 2023-07-06T02:25:29.575 INFO:teuthology.orchestra.run.smithi089.stdout:522.out 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:523 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:523.out 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:524 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:524.out 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:525 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:525.out 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:526 2023-07-06T02:25:29.576 INFO:teuthology.orchestra.run.smithi089.stdout:526.out 2023-07-06T02:25:29.577 INFO:teuthology.orchestra.run.smithi089.stdout:527 2023-07-06T02:25:29.577 INFO:teuthology.orchestra.run.smithi089.stdout:527.out 2023-07-06T02:25:29.577 INFO:teuthology.orchestra.run.smithi089.stdout:528 2023-07-06T02:25:29.577 INFO:teuthology.orchestra.run.smithi089.stdout:528.out 2023-07-06T02:25:29.577 INFO:teuthology.orchestra.run.smithi089.stdout:529 2023-07-06T02:25:29.577 INFO:teuthology.orchestra.run.smithi089.stdout:529.out 2023-07-06T02:25:29.577 INFO:teuthology.orchestra.run.smithi089.stdout:530 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:530.out 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:531 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:531.out 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:532 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:532.out 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:533 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:533.out 2023-07-06T02:25:29.578 INFO:teuthology.orchestra.run.smithi089.stdout:534 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:534.out 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:535 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:535.out 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:536 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:536.out 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:537 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:537.out 2023-07-06T02:25:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:538 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:538.out 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:539 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:539.out 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:540 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:540.out 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:541 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:541.out 2023-07-06T02:25:29.580 INFO:teuthology.orchestra.run.smithi089.stdout:542 2023-07-06T02:25:29.581 INFO:teuthology.orchestra.run.smithi089.stdout:542.out 2023-07-06T02:25:29.581 INFO:teuthology.orchestra.run.smithi089.stdout:543 2023-07-06T02:25:29.581 INFO:teuthology.orchestra.run.smithi089.stdout:543.out 2023-07-06T02:25:29.581 INFO:teuthology.orchestra.run.smithi089.stdout:544 2023-07-06T02:25:29.581 INFO:teuthology.orchestra.run.smithi089.stdout:544.out 2023-07-06T02:25:29.581 INFO:teuthology.orchestra.run.smithi089.stdout:545 2023-07-06T02:25:29.581 INFO:teuthology.orchestra.run.smithi089.stdout:545.out 2023-07-06T02:25:29.582 INFO:teuthology.orchestra.run.smithi089.stdout:546 2023-07-06T02:25:29.582 INFO:teuthology.orchestra.run.smithi089.stdout:546.out 2023-07-06T02:25:29.582 INFO:teuthology.orchestra.run.smithi089.stdout:547 2023-07-06T02:25:29.582 INFO:teuthology.orchestra.run.smithi089.stdout:547.out 2023-07-06T02:25:29.582 INFO:teuthology.orchestra.run.smithi089.stdout:548 2023-07-06T02:25:29.582 INFO:teuthology.orchestra.run.smithi089.stdout:548.out 2023-07-06T02:25:29.582 INFO:teuthology.orchestra.run.smithi089.stdout:549 2023-07-06T02:25:29.583 INFO:teuthology.orchestra.run.smithi089.stdout:549.out 2023-07-06T02:25:29.583 INFO:teuthology.orchestra.run.smithi089.stdout:550 2023-07-06T02:25:29.583 INFO:teuthology.orchestra.run.smithi089.stdout:550.out 2023-07-06T02:25:29.583 INFO:teuthology.orchestra.run.smithi089.stdout:551 2023-07-06T02:25:29.583 INFO:teuthology.orchestra.run.smithi089.stdout:551.out 2023-07-06T02:25:29.583 INFO:teuthology.orchestra.run.smithi089.stdout:552 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:552.out 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:553 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:553.out 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:554 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:554.out 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:555 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:555.out 2023-07-06T02:25:29.584 INFO:teuthology.orchestra.run.smithi089.stdout:556 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:556.out 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:557 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:557.out 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:558 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:558.out 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:559 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:559.out 2023-07-06T02:25:29.585 INFO:teuthology.orchestra.run.smithi089.stdout:560 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:560.out 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:561 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:561.out 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:562 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:562.out 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:563 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:563.out 2023-07-06T02:25:29.586 INFO:teuthology.orchestra.run.smithi089.stdout:564 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:564.out 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:565 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:565.out 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:566 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:566.out 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:567 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:567.out 2023-07-06T02:25:29.587 INFO:teuthology.orchestra.run.smithi089.stdout:568 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:568.out 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:569 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:569.out 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:570 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:570.out 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:571 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:571.out 2023-07-06T02:25:29.588 INFO:teuthology.orchestra.run.smithi089.stdout:572 2023-07-06T02:25:29.589 INFO:teuthology.orchestra.run.smithi089.stdout:572.out 2023-07-06T02:25:29.589 INFO:teuthology.orchestra.run.smithi089.stdout:573 2023-07-06T02:25:29.589 INFO:teuthology.orchestra.run.smithi089.stdout:573.out 2023-07-06T02:25:29.589 INFO:teuthology.orchestra.run.smithi089.stdout:574 2023-07-06T02:25:29.589 INFO:teuthology.orchestra.run.smithi089.stdout:574.out 2023-07-06T02:25:29.589 INFO:teuthology.orchestra.run.smithi089.stdout:575 2023-07-06T02:25:29.589 INFO:teuthology.orchestra.run.smithi089.stdout:575.out 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:576 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:576.out 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:577 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:577.out 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:578 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:578.out 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:579 2023-07-06T02:25:29.590 INFO:teuthology.orchestra.run.smithi089.stdout:579.out 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:580 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:580.out 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:581 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:581.out 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:582 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:582.out 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:583 2023-07-06T02:25:29.591 INFO:teuthology.orchestra.run.smithi089.stdout:583.out 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:584 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:584.out 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:585 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:585.out 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:586 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:586.out 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:587 2023-07-06T02:25:29.592 INFO:teuthology.orchestra.run.smithi089.stdout:587.out 2023-07-06T02:25:29.593 INFO:teuthology.orchestra.run.smithi089.stdout:588 2023-07-06T02:25:29.593 INFO:teuthology.orchestra.run.smithi089.stdout:588.out 2023-07-06T02:25:29.593 INFO:teuthology.orchestra.run.smithi089.stdout:589 2023-07-06T02:25:29.593 INFO:teuthology.orchestra.run.smithi089.stdout:589.out 2023-07-06T02:25:29.593 INFO:teuthology.orchestra.run.smithi089.stdout:590 2023-07-06T02:25:29.593 INFO:teuthology.orchestra.run.smithi089.stdout:590.out 2023-07-06T02:25:29.593 INFO:teuthology.orchestra.run.smithi089.stdout:591 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:591.out 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:592 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:592.out 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:593 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:593.out 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:594 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:594.out 2023-07-06T02:25:29.594 INFO:teuthology.orchestra.run.smithi089.stdout:595 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:595.out 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:596 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:596.out 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:597 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:597.out 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:598 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:598.out 2023-07-06T02:25:29.595 INFO:teuthology.orchestra.run.smithi089.stdout:599 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:599.out 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:600 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:600.out 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:601 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:601.out 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:602 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:602.out 2023-07-06T02:25:29.596 INFO:teuthology.orchestra.run.smithi089.stdout:603 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:603.out 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:604 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:604.out 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:605 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:605.out 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:606 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:606.out 2023-07-06T02:25:29.597 INFO:teuthology.orchestra.run.smithi089.stdout:607 2023-07-06T02:25:29.598 INFO:teuthology.orchestra.run.smithi089.stdout:607.out 2023-07-06T02:25:29.598 INFO:teuthology.orchestra.run.smithi089.stdout:608 2023-07-06T02:25:29.598 INFO:teuthology.orchestra.run.smithi089.stdout:608.out 2023-07-06T02:25:29.598 INFO:teuthology.orchestra.run.smithi089.stdout:609 2023-07-06T02:25:29.598 INFO:teuthology.orchestra.run.smithi089.stdout:609.out 2023-07-06T02:25:29.598 INFO:teuthology.orchestra.run.smithi089.stdout:610 2023-07-06T02:25:29.598 INFO:teuthology.orchestra.run.smithi089.stdout:610.out 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:611 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:611.out 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:612 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:612.out 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:613 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:613.out 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:614 2023-07-06T02:25:29.599 INFO:teuthology.orchestra.run.smithi089.stdout:614.out 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:615 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:615.out 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:616 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:616.out 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:617 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:617.out 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:618 2023-07-06T02:25:29.600 INFO:teuthology.orchestra.run.smithi089.stdout:618.out 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:619 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:619.out 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:620 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:620.out 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:621 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:621.out 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:622 2023-07-06T02:25:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:622.out 2023-07-06T02:25:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:623 2023-07-06T02:25:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:623.out 2023-07-06T02:25:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:624 2023-07-06T02:25:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:624.out 2023-07-06T02:25:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:625 2023-07-06T02:25:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:625.out 2023-07-06T02:25:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:626 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:626.out 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:627 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:627.out 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:628 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:628.out 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:629 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:629.out 2023-07-06T02:25:29.603 INFO:teuthology.orchestra.run.smithi089.stdout:630 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:630.out 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:631 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:631.out 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:632 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:632.out 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:633 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:633.out 2023-07-06T02:25:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:634 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:634.out 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:635 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:635.out 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:636 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:636.out 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:637 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:637.out 2023-07-06T02:25:29.605 INFO:teuthology.orchestra.run.smithi089.stdout:638 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:638.out 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:639 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:639.out 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:640 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:640.out 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:641 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:641.out 2023-07-06T02:25:29.606 INFO:teuthology.orchestra.run.smithi089.stdout:642 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:642.out 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:643 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:643.out 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:644 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:644.out 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:645 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:645.out 2023-07-06T02:25:29.607 INFO:teuthology.orchestra.run.smithi089.stdout:646 2023-07-06T02:25:29.608 INFO:teuthology.orchestra.run.smithi089.stdout:646.out 2023-07-06T02:25:29.608 INFO:teuthology.orchestra.run.smithi089.stdout:647 2023-07-06T02:25:29.608 INFO:teuthology.orchestra.run.smithi089.stdout:647.out 2023-07-06T02:25:29.608 INFO:teuthology.orchestra.run.smithi089.stdout:648 2023-07-06T02:25:29.608 INFO:teuthology.orchestra.run.smithi089.stdout:648.out 2023-07-06T02:25:29.608 INFO:teuthology.orchestra.run.smithi089.stdout:649 2023-07-06T02:25:29.608 INFO:teuthology.orchestra.run.smithi089.stdout:649.out 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:650 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:650.out 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:651 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:651.out 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:652 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:652.out 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:653 2023-07-06T02:25:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:653.out 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:654 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:654.out 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:655 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:655.out 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:656 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:656.out 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:657 2023-07-06T02:25:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:657.out 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:658 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:658.out 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:659 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:659.out 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:660 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:660.out 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:661 2023-07-06T02:25:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:661.out 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:662 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:662.out 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:663 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:663.out 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:664 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:664.out 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:665 2023-07-06T02:25:29.612 INFO:teuthology.orchestra.run.smithi089.stdout:665.out 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:666 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:666.out 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:667 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:667.out 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:668 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:668.out 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:669 2023-07-06T02:25:29.613 INFO:teuthology.orchestra.run.smithi089.stdout:669.out 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:670 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:670.out 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:671 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:671.out 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:672 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:672.out 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:673 2023-07-06T02:25:29.614 INFO:teuthology.orchestra.run.smithi089.stdout:673.out 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:674 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:674.out 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:675 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:675.out 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:676 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:676.out 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:677 2023-07-06T02:25:29.615 INFO:teuthology.orchestra.run.smithi089.stdout:677.out 2023-07-06T02:25:29.616 INFO:teuthology.orchestra.run.smithi089.stdout:678 2023-07-06T02:25:29.616 INFO:teuthology.orchestra.run.smithi089.stdout:678.out 2023-07-06T02:25:29.616 INFO:teuthology.orchestra.run.smithi089.stdout:679 2023-07-06T02:25:29.616 INFO:teuthology.orchestra.run.smithi089.stdout:679.out 2023-07-06T02:25:29.616 INFO:teuthology.orchestra.run.smithi089.stdout:680 2023-07-06T02:25:29.616 INFO:teuthology.orchestra.run.smithi089.stdout:680.out 2023-07-06T02:25:29.616 INFO:teuthology.orchestra.run.smithi089.stdout:681 2023-07-06T02:25:29.617 INFO:teuthology.orchestra.run.smithi089.stdout:681.out 2023-07-06T02:25:29.617 INFO:teuthology.orchestra.run.smithi089.stdout:682 2023-07-06T02:25:29.617 INFO:teuthology.orchestra.run.smithi089.stdout:682.out 2023-07-06T02:25:29.617 INFO:teuthology.orchestra.run.smithi089.stdout:683 2023-07-06T02:25:29.617 INFO:teuthology.orchestra.run.smithi089.stdout:683.out 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:684 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:684.out 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:685 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:685.out 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:686 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:686.out 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:687 2023-07-06T02:25:29.618 INFO:teuthology.orchestra.run.smithi089.stdout:687.out 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:688 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:688.out 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:689 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:689.out 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:690 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:690.out 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:691 2023-07-06T02:25:29.619 INFO:teuthology.orchestra.run.smithi089.stdout:691.out 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:692 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:692.out 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:693 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:693.out 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:694 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:694.out 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:695 2023-07-06T02:25:29.620 INFO:teuthology.orchestra.run.smithi089.stdout:695.out 2023-07-06T02:25:29.621 INFO:teuthology.orchestra.run.smithi089.stdout:696 2023-07-06T02:25:29.621 INFO:teuthology.orchestra.run.smithi089.stdout:696.out 2023-07-06T02:25:29.621 INFO:teuthology.orchestra.run.smithi089.stdout:697 2023-07-06T02:25:29.621 INFO:teuthology.orchestra.run.smithi089.stdout:697.out 2023-07-06T02:25:29.621 INFO:teuthology.orchestra.run.smithi089.stdout:698 2023-07-06T02:25:29.621 INFO:teuthology.orchestra.run.smithi089.stdout:698.out 2023-07-06T02:25:29.621 INFO:teuthology.orchestra.run.smithi089.stdout:699 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:699.out 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:700 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:700.out 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:701 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:701.out 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:702 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:702.out 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:703 2023-07-06T02:25:29.622 INFO:teuthology.orchestra.run.smithi089.stdout:703.out 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:704 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:704.out 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:705 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:705.out 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:706 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:706.out 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:707 2023-07-06T02:25:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:707.out 2023-07-06T02:25:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:708 2023-07-06T02:25:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:708.out 2023-07-06T02:25:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:709 2023-07-06T02:25:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:709.out 2023-07-06T02:25:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:710 2023-07-06T02:25:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:710.out 2023-07-06T02:25:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:711 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:711.out 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:712 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:712.out 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:713 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:713.out 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:714 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:714.out 2023-07-06T02:25:29.625 INFO:teuthology.orchestra.run.smithi089.stdout:715 2023-07-06T02:25:29.626 INFO:teuthology.orchestra.run.smithi089.stdout:715.out 2023-07-06T02:25:29.626 INFO:teuthology.orchestra.run.smithi089.stdout:716 2023-07-06T02:25:29.626 INFO:teuthology.orchestra.run.smithi089.stdout:716.out 2023-07-06T02:25:29.626 INFO:teuthology.orchestra.run.smithi089.stdout:717 2023-07-06T02:25:29.626 INFO:teuthology.orchestra.run.smithi089.stdout:717.out 2023-07-06T02:25:29.626 INFO:teuthology.orchestra.run.smithi089.stdout:718 2023-07-06T02:25:29.626 INFO:teuthology.orchestra.run.smithi089.stdout:718.out 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:719 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:719.out 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:720 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:720.out 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:721 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:721.out 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:722 2023-07-06T02:25:29.627 INFO:teuthology.orchestra.run.smithi089.stdout:722.out 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:723 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:723.out 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:724 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:724.out 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:725 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:725.out 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:726 2023-07-06T02:25:29.628 INFO:teuthology.orchestra.run.smithi089.stdout:726.out 2023-07-06T02:25:29.629 INFO:teuthology.orchestra.run.smithi089.stdout:727 2023-07-06T02:25:29.629 INFO:teuthology.orchestra.run.smithi089.stdout:727.out 2023-07-06T02:25:29.629 INFO:teuthology.orchestra.run.smithi089.stdout:728 2023-07-06T02:25:29.629 INFO:teuthology.orchestra.run.smithi089.stdout:728.out 2023-07-06T02:25:29.629 INFO:teuthology.orchestra.run.smithi089.stdout:group.list 2023-07-06T02:25:29.629 INFO:teuthology.orchestra.run.smithi089.stdout:Makefile 2023-07-06T02:25:29.630 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-07-06T02:25:29.852 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:25:29.856 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:25:29.856 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:25:29.857 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:25:29.857 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:34.535 INFO:teuthology.orchestra.run.smithi089.stdout:generic/001 4s 2023-07-06T02:25:34.540 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/001 2023-07-06T02:25:34.541 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:25:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:35.244 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:25:35.245 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-07-06T02:25:35.451 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:25:35.455 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:25:35.455 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:25:35.456 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:25:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:36.075 INFO:teuthology.orchestra.run.smithi089.stdout:generic/002 1s 2023-07-06T02:25:36.079 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/002 2023-07-06T02:25:36.079 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:25:36.081 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:36.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:25:36.149 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-07-06T02:25:36.367 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:25:36.371 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:25:36.371 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:25:36.372 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:25:36.372 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:36.661 INFO:teuthology.orchestra.run.smithi089.stdout:generic/003 [not run] atime not maintained by ceph 2023-07-06T02:25:36.665 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/003 2023-07-06T02:25:36.666 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/003 2023-07-06T02:25:36.666 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:25:36.667 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:36.687 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:25:36.687 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-07-06T02:25:36.896 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:25:36.900 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:25:36.900 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:25:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:25:36.901 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:37.204 INFO:teuthology.orchestra.run.smithi089.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-07-06T02:25:37.208 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/004 2023-07-06T02:25:37.208 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/004 2023-07-06T02:25:37.208 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:25:37.210 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:37.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:25:37.233 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-07-06T02:25:37.440 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:25:37.444 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:25:37.445 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:25:37.445 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:25:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:37.896 INFO:teuthology.orchestra.run.smithi089.stdout:generic/005 0s 2023-07-06T02:25:37.901 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/005 2023-07-06T02:25:37.901 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:25:37.903 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:38.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:25:38.007 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-07-06T02:25:38.201 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:25:38.205 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:25:38.205 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:25:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:25:38.206 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:25:59.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:25:59.635 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:26:02.338 INFO:teuthology.orchestra.run.smithi089.stdout:generic/006 24s 2023-07-06T02:26:02.342 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/006 2023-07-06T02:26:02.343 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:26:02.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:02.409 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:26:02.409 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-07-06T02:26:02.606 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:26:02.610 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:26:02.610 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:26:02.611 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:26:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:29.714 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:26:29.716 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:26:51.937 INFO:teuthology.orchestra.run.smithi089.stdout:generic/007 49s 2023-07-06T02:26:51.941 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/007 2023-07-06T02:26:51.941 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:26:51.943 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:52.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:26:52.034 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-07-06T02:26:52.229 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:26:52.233 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:26:52.233 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:26:52.234 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:26:52.235 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:52.642 INFO:teuthology.orchestra.run.smithi089.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-06T02:26:52.646 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/008 2023-07-06T02:26:52.646 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/008 2023-07-06T02:26:52.647 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:26:52.648 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:52.672 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:26:52.673 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-07-06T02:26:52.866 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:26:52.870 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:26:52.871 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:26:52.871 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:26:52.871 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:53.247 INFO:teuthology.orchestra.run.smithi089.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-06T02:26:53.251 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/009 2023-07-06T02:26:53.251 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/009 2023-07-06T02:26:53.251 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:26:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:53.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:26:53.274 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-07-06T02:26:53.472 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:26:53.476 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:26:53.477 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:26:53.478 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:26:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:53.756 INFO:teuthology.orchestra.run.smithi089.stdout:generic/010 [not run] /tmp/tmp.KkaxlHRTc5xfstests-dev/src/dbtest not built 2023-07-06T02:26:53.760 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/010 2023-07-06T02:26:53.760 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/010 2023-07-06T02:26:53.760 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:26:53.762 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:26:53.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:26:53.783 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-07-06T02:26:53.986 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:26:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:26:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:26:53.991 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:26:53.992 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:27:00.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:27:00.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:27:24.132 INFO:teuthology.orchestra.run.smithi089.stdout:generic/011 30s 2023-07-06T02:27:24.201 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/011 2023-07-06T02:27:24.201 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:27:24.203 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:27:24.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:27:24.452 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-07-06T02:27:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:27:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:27:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:27:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:27:24.672 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:27:25.152 INFO:teuthology.orchestra.run.smithi089.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:27:25.155 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/012 2023-07-06T02:27:25.156 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/012 2023-07-06T02:27:25.156 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:27:25.223 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:27:25.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:27:25.296 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-07-06T02:27:25.491 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:27:25.495 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:27:25.495 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:27:25.496 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:27:25.496 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:27:30.131 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:27:30.134 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:28:00.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:28:00.472 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:28:29.122 INFO:teuthology.orchestra.run.smithi089.stdout:generic/013 64s 2023-07-06T02:28:29.131 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/013 2023-07-06T02:28:29.132 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:28:29.134 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:28:29.167 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:28:29.167 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-07-06T02:28:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:28:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:28:29.495 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:28:29.496 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:28:29.497 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:28:30.830 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:28:30.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:29:01.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:29:01.132 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:29:31.459 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:29:31.463 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:30:01.798 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:30:01.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:30:31.888 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:30:31.897 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:31:02.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:31:02.293 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:31:32.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:31:32.686 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:32:03.138 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:32:03.141 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:32:33.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:32:33.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:33:03.324 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:33:03.327 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:33:16.522 INFO:teuthology.orchestra.run.smithi089.stdout:generic/014 287s 2023-07-06T02:33:16.527 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/014 2023-07-06T02:33:16.527 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:16.535 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:16.588 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:16.589 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2023-07-06T02:33:16.946 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:16.954 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:16.954 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:16.957 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:16.957 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:17.439 INFO:teuthology.orchestra.run.smithi089.stdout:generic/015 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-06T02:33:17.444 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/015 2023-07-06T02:33:17.444 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/015 2023-07-06T02:33:17.444 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:17.446 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:17.474 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:17.475 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2023-07-06T02:33:17.832 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:17.837 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:17.838 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:17.839 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:18.456 INFO:teuthology.orchestra.run.smithi089.stdout:generic/016 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:33:18.463 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/016 2023-07-06T02:33:18.463 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/016 2023-07-06T02:33:18.463 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:18.465 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:18.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:18.697 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2023-07-06T02:33:19.145 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:19.154 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:19.154 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:19.158 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:19.158 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:19.640 INFO:teuthology.orchestra.run.smithi089.stdout:generic/017 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-06T02:33:19.644 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/017 2023-07-06T02:33:19.644 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/017 2023-07-06T02:33:19.645 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:19.646 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:19.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:19.680 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2023-07-06T02:33:20.009 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:20.013 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:20.014 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:20.015 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:20.016 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:20.641 INFO:teuthology.orchestra.run.smithi089.stdout:generic/018 [not run] defragmentation not supported for fstype "ceph" 2023-07-06T02:33:20.661 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/018 2023-07-06T02:33:20.661 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/018 2023-07-06T02:33:20.661 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:20.664 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:20.686 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:20.687 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2023-07-06T02:33:21.006 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:21.011 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:21.012 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:21.013 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:21.013 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:21.466 INFO:teuthology.orchestra.run.smithi089.stdout:generic/019 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:33:21.470 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/019 2023-07-06T02:33:21.471 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/019 2023-07-06T02:33:21.471 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:21.473 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:21.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:21.505 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2023-07-06T02:33:21.875 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:21.890 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:21.890 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:21.899 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:21.899 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:25.454 INFO:teuthology.orchestra.run.smithi089.stdout:generic/020 3s 2023-07-06T02:33:25.460 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/020 2023-07-06T02:33:25.461 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:25.754 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:25.754 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2023-07-06T02:33:26.112 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:26.116 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:26.117 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:26.117 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:26.118 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:26.751 INFO:teuthology.orchestra.run.smithi089.stdout:generic/021 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:33:26.756 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/021 2023-07-06T02:33:26.757 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/021 2023-07-06T02:33:26.757 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:26.760 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:27.129 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:27.130 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2023-07-06T02:33:27.458 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:27.472 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:27.472 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:27.473 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:28.083 INFO:teuthology.orchestra.run.smithi089.stdout:generic/022 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:33:28.093 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/022 2023-07-06T02:33:28.093 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/022 2023-07-06T02:33:28.093 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:33:28.095 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:28.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:33:28.342 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2023-07-06T02:33:28.823 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:33:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:33:28.829 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:33:28.830 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:33:28.830 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:33:33.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:33:33.495 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:34:03.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:34:03.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:34:22.965 INFO:teuthology.orchestra.run.smithi089.stdout:generic/023 53s 2023-07-06T02:34:22.970 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/023 2023-07-06T02:34:22.970 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:23.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:23.372 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2023-07-06T02:34:23.723 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:23.737 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:23.737 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:23.738 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:24.289 INFO:teuthology.orchestra.run.smithi089.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2023-07-06T02:34:24.293 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/024 2023-07-06T02:34:24.293 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/024 2023-07-06T02:34:24.293 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:24.295 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:24.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:24.672 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2023-07-06T02:34:25.029 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:25.033 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:25.034 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:25.035 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:25.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:25.537 INFO:teuthology.orchestra.run.smithi089.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2023-07-06T02:34:25.545 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/025 2023-07-06T02:34:25.545 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/025 2023-07-06T02:34:25.546 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:25.548 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:25.926 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:25.926 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2023-07-06T02:34:26.245 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:26.250 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:26.250 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:26.252 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:26.252 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:26.822 INFO:teuthology.orchestra.run.smithi089.stdout:generic/026 [not run] ceph does not define maximum ACL count 2023-07-06T02:34:26.825 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/026 2023-07-06T02:34:26.825 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/026 2023-07-06T02:34:26.825 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:26.828 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:26.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:26.877 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2023-07-06T02:34:27.250 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:27.253 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:27.254 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:27.256 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:27.256 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:27.706 INFO:teuthology.orchestra.run.smithi089.stdout:generic/027 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-06T02:34:27.711 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/027 2023-07-06T02:34:27.711 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/027 2023-07-06T02:34:27.712 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:27.714 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:27.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:27.740 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2023-07-06T02:34:28.089 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:28.096 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:28.096 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:28.098 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:28.098 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:33.584 INFO:teuthology.orchestra.run.smithi089.stdout:generic/028 5s 2023-07-06T02:34:33.589 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/028 2023-07-06T02:34:33.590 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:33.595 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:33.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:33.634 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2023-07-06T02:34:33.811 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:34:33.814 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:34:34.020 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:34.025 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:34.026 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:34.028 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:34.028 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:40.260 INFO:teuthology.orchestra.run.smithi089.stdout:generic/029 6s 2023-07-06T02:34:40.264 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/029 2023-07-06T02:34:40.265 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:40.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:40.275 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2023-07-06T02:34:40.475 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:40.479 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:40.480 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:40.481 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:40.481 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:43.180 INFO:teuthology.orchestra.run.smithi089.stdout:generic/030 3s 2023-07-06T02:34:43.184 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/030 2023-07-06T02:34:43.185 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:43.186 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:43.194 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:43.195 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2023-07-06T02:34:43.389 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:43.393 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:43.394 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:43.395 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:43.396 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:43.782 INFO:teuthology.orchestra.run.smithi089.stdout:generic/031 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-07-06T02:34:43.786 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/031 2023-07-06T02:34:43.786 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/031 2023-07-06T02:34:43.786 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:43.788 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:43.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:43.816 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2023-07-06T02:34:44.015 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:44.019 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:44.019 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:44.020 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:44.020 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:44.343 INFO:teuthology.orchestra.run.smithi089.stdout:generic/032 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:34:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/032 2023-07-06T02:34:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/032 2023-07-06T02:34:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:44.349 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:44.376 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:44.376 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2023-07-06T02:34:44.582 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:44.587 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:44.979 INFO:teuthology.orchestra.run.smithi089.stdout:generic/033 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-06T02:34:44.983 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/033 2023-07-06T02:34:44.983 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/033 2023-07-06T02:34:44.983 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:44.985 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:45.011 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:45.012 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2023-07-06T02:34:45.235 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:45.239 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:45.239 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:45.240 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:45.240 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:45.556 INFO:teuthology.orchestra.run.smithi089.stdout:generic/034 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:34:45.560 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/034 2023-07-06T02:34:45.561 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/034 2023-07-06T02:34:45.561 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:45.562 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:45.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:45.596 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2023-07-06T02:34:45.794 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:45.798 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:45.798 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:45.799 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:45.799 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:46.148 INFO:teuthology.orchestra.run.smithi089.stdout:generic/035 1s 2023-07-06T02:34:46.152 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/035 2023-07-06T02:34:46.153 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:46.154 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:46.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:46.511 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2023-07-06T02:34:46.709 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:46.713 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:46.714 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:46.714 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:46.715 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:57.083 INFO:teuthology.orchestra.run.smithi089.stdout:generic/036 11s 2023-07-06T02:34:57.088 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/036 2023-07-06T02:34:57.088 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:34:57.090 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:34:57.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:34:57.375 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2023-07-06T02:34:57.632 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:34:57.635 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:34:57.636 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:34:57.637 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:34:57.637 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:00.125 INFO:teuthology.orchestra.run.smithi089.stdout:generic/037 3s 2023-07-06T02:35:00.130 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/037 2023-07-06T02:35:00.130 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:00.139 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:00.140 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2023-07-06T02:35:00.336 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:00.340 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:00.340 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:00.341 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:00.654 INFO:teuthology.orchestra.run.smithi089.stdout:generic/038 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:35:00.658 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/038 2023-07-06T02:35:00.659 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/038 2023-07-06T02:35:00.659 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:00.687 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:00.687 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2023-07-06T02:35:00.894 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:00.898 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:00.898 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:00.899 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:01.223 INFO:teuthology.orchestra.run.smithi089.stdout:generic/039 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:01.227 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/039 2023-07-06T02:35:01.227 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/039 2023-07-06T02:35:01.227 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:01.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:01.254 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2023-07-06T02:35:01.458 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:01.462 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:01.463 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:01.463 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:01.799 INFO:teuthology.orchestra.run.smithi089.stdout:generic/040 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:01.803 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/040 2023-07-06T02:35:01.803 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/040 2023-07-06T02:35:01.803 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:01.805 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:01.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:01.829 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2023-07-06T02:35:02.023 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:02.027 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:02.028 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:02.028 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:02.029 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:02.356 INFO:teuthology.orchestra.run.smithi089.stdout:generic/041 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:02.360 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/041 2023-07-06T02:35:02.360 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/041 2023-07-06T02:35:02.361 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:02.387 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:02.388 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2023-07-06T02:35:02.602 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:02.606 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:02.606 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:02.607 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:03.093 INFO:teuthology.orchestra.run.smithi089.stdout:generic/042 [not run] ceph does not support shutdown 2023-07-06T02:35:03.108 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/042 2023-07-06T02:35:03.108 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/042 2023-07-06T02:35:03.108 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:03.110 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:03.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:03.138 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2023-07-06T02:35:03.333 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:03.337 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:03.337 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:03.338 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:03.338 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:03.759 INFO:teuthology.orchestra.run.smithi089.stdout:generic/043 [not run] ceph does not support shutdown 2023-07-06T02:35:03.784 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/043 2023-07-06T02:35:03.785 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/043 2023-07-06T02:35:03.785 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:03.786 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:03.806 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:03.807 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2023-07-06T02:35:04.000 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:04.004 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:04.005 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:04.006 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:04.006 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:04.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:35:04.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:35:04.489 INFO:teuthology.orchestra.run.smithi089.stdout:generic/044 [not run] ceph does not support shutdown 2023-07-06T02:35:04.511 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/044 2023-07-06T02:35:04.511 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/044 2023-07-06T02:35:04.511 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:04.512 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:04.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:04.539 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2023-07-06T02:35:04.731 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:04.735 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:04.736 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:04.737 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:05.183 INFO:teuthology.orchestra.run.smithi089.stdout:generic/045 [not run] ceph does not support shutdown 2023-07-06T02:35:05.209 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/045 2023-07-06T02:35:05.210 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/045 2023-07-06T02:35:05.210 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:05.211 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:05.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:05.234 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2023-07-06T02:35:05.428 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:05.429 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:05.430 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:05.431 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:05.431 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:05.906 INFO:teuthology.orchestra.run.smithi089.stdout:generic/046 [not run] ceph does not support shutdown 2023-07-06T02:35:05.926 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/046 2023-07-06T02:35:05.926 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/046 2023-07-06T02:35:05.926 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:05.928 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:05.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:05.954 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2023-07-06T02:35:06.158 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:06.162 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:06.163 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:06.164 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:06.164 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:06.619 INFO:teuthology.orchestra.run.smithi089.stdout:generic/047 [not run] ceph does not support shutdown 2023-07-06T02:35:06.649 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/047 2023-07-06T02:35:06.650 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/047 2023-07-06T02:35:06.650 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:06.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:06.671 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2023-07-06T02:35:06.868 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:06.872 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:06.873 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:06.874 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:06.874 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:07.324 INFO:teuthology.orchestra.run.smithi089.stdout:generic/048 [not run] ceph does not support shutdown 2023-07-06T02:35:07.341 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/048 2023-07-06T02:35:07.341 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/048 2023-07-06T02:35:07.341 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:07.343 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:07.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:07.377 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2023-07-06T02:35:07.571 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:07.575 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:07.576 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:07.577 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:07.577 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:08.005 INFO:teuthology.orchestra.run.smithi089.stdout:generic/049 [not run] ceph does not support shutdown 2023-07-06T02:35:08.024 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/049 2023-07-06T02:35:08.025 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/049 2023-07-06T02:35:08.025 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:08.026 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:08.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:08.049 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2023-07-06T02:35:08.250 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:08.254 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:08.254 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:08.255 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:08.255 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:08.711 INFO:teuthology.orchestra.run.smithi089.stdout:generic/050 [not run] ceph does not support shutdown 2023-07-06T02:35:08.726 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/050 2023-07-06T02:35:08.727 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/050 2023-07-06T02:35:08.727 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:08.764 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:08.765 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2023-07-06T02:35:08.959 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:08.963 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:08.963 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:08.964 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:08.964 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:09.438 INFO:teuthology.orchestra.run.smithi089.stdout:generic/051 [not run] ceph does not support shutdown 2023-07-06T02:35:09.442 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/051 2023-07-06T02:35:09.443 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/051 2023-07-06T02:35:09.443 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:09.445 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:09.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:09.468 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2023-07-06T02:35:09.669 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:09.672 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:09.673 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:09.674 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:09.674 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:10.147 INFO:teuthology.orchestra.run.smithi089.stdout:generic/052 [not run] ceph does not support shutdown 2023-07-06T02:35:10.161 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/052 2023-07-06T02:35:10.162 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/052 2023-07-06T02:35:10.162 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:10.163 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:10.188 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:10.188 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2023-07-06T02:35:10.381 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:10.385 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:10.386 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:10.386 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:10.387 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:11.311 INFO:teuthology.orchestra.run.smithi089.stdout:generic/053 1s 2023-07-06T02:35:11.336 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/053 2023-07-06T02:35:11.336 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:11.338 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:11.347 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:11.348 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2023-07-06T02:35:11.556 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:11.560 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:11.560 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:11.561 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:12.025 INFO:teuthology.orchestra.run.smithi089.stdout:generic/054 [not run] ceph does not support shutdown 2023-07-06T02:35:12.037 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/054 2023-07-06T02:35:12.037 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/054 2023-07-06T02:35:12.038 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:12.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:12.059 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2023-07-06T02:35:12.260 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:12.264 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:12.265 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:12.265 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:12.266 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:12.739 INFO:teuthology.orchestra.run.smithi089.stdout:generic/055 [not run] ceph does not support shutdown 2023-07-06T02:35:12.751 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/055 2023-07-06T02:35:12.752 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/055 2023-07-06T02:35:12.752 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:12.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:12.778 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2023-07-06T02:35:12.971 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:12.975 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:12.976 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:12.977 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:12.977 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:13.301 INFO:teuthology.orchestra.run.smithi089.stdout:generic/056 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:13.305 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/056 2023-07-06T02:35:13.305 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/056 2023-07-06T02:35:13.306 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:13.307 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:13.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:13.333 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2023-07-06T02:35:13.529 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:13.533 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:13.533 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:13.534 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:13.534 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:13.864 INFO:teuthology.orchestra.run.smithi089.stdout:generic/057 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:13.868 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/057 2023-07-06T02:35:13.868 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/057 2023-07-06T02:35:13.868 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:13.870 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:13.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:13.895 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2023-07-06T02:35:14.091 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:14.095 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:14.096 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:14.097 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:14.522 INFO:teuthology.orchestra.run.smithi089.stdout:generic/058 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:35:14.526 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/058 2023-07-06T02:35:14.526 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/058 2023-07-06T02:35:14.526 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:14.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:14.873 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2023-07-06T02:35:15.075 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:15.079 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:15.080 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:15.080 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:15.393 INFO:teuthology.orchestra.run.smithi089.stdout:generic/059 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:15.397 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/059 2023-07-06T02:35:15.397 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/059 2023-07-06T02:35:15.398 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:15.399 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:15.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:15.427 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2023-07-06T02:35:15.628 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:15.632 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:15.633 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:15.633 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:15.633 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:16.036 INFO:teuthology.orchestra.run.smithi089.stdout:generic/060 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:35:16.040 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/060 2023-07-06T02:35:16.040 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/060 2023-07-06T02:35:16.040 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:16.042 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:16.385 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:16.385 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2023-07-06T02:35:16.626 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:16.630 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:16.630 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:16.631 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:16.631 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:17.027 INFO:teuthology.orchestra.run.smithi089.stdout:generic/061 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:35:17.031 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/061 2023-07-06T02:35:17.031 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/061 2023-07-06T02:35:17.031 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:17.033 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:17.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:17.362 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2023-07-06T02:35:17.592 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:17.596 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:17.597 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:17.597 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:19.369 INFO:teuthology.orchestra.run.smithi089.stdout:generic/062 2s 2023-07-06T02:35:19.376 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/062 2023-07-06T02:35:19.377 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:19.379 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:19.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:19.391 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2023-07-06T02:35:19.609 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:19.613 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:19.614 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:19.614 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:19.615 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:20.037 INFO:teuthology.orchestra.run.smithi089.stdout:generic/063 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:35:20.041 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/063 2023-07-06T02:35:20.042 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/063 2023-07-06T02:35:20.042 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:20.043 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:20.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:20.324 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2023-07-06T02:35:20.616 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:20.620 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:20.620 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:20.621 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:20.621 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:20.928 INFO:teuthology.orchestra.run.smithi089.stdout:generic/064 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-06T02:35:20.932 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/064 2023-07-06T02:35:20.932 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/064 2023-07-06T02:35:20.932 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:20.934 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:20.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:20.963 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2023-07-06T02:35:21.158 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:21.162 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:21.163 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:21.164 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:21.490 INFO:teuthology.orchestra.run.smithi089.stdout:generic/065 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:21.493 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/065 2023-07-06T02:35:21.494 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/065 2023-07-06T02:35:21.494 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:21.496 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:21.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:21.526 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2023-07-06T02:35:21.736 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:21.740 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:21.740 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:21.741 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:21.741 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:22.066 INFO:teuthology.orchestra.run.smithi089.stdout:generic/066 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:22.070 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/066 2023-07-06T02:35:22.071 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/066 2023-07-06T02:35:22.071 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:22.072 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:22.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:22.092 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2023-07-06T02:35:22.286 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:22.290 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:22.290 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:22.292 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:22.292 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:22.636 INFO:teuthology.orchestra.run.smithi089.stdout:generic/067 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:22.639 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/067 2023-07-06T02:35:22.640 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/067 2023-07-06T02:35:22.640 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:22.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:22.668 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2023-07-06T02:35:22.867 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:22.870 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:22.871 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:22.872 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:23.176 INFO:teuthology.orchestra.run.smithi089.stdout:generic/068 [not run] ceph does not support freezing 2023-07-06T02:35:23.180 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/068 2023-07-06T02:35:23.180 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/068 2023-07-06T02:35:23.180 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:23.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:23.208 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2023-07-06T02:35:23.404 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:23.408 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:23.408 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:23.409 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:28.997 INFO:teuthology.orchestra.run.smithi089.stdout:generic/069 5s 2023-07-06T02:35:29.002 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/069 2023-07-06T02:35:29.002 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:29.004 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:29.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:29.013 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2023-07-06T02:35:29.206 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:29.210 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:29.210 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:29.211 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:29.211 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:34.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:35:34.298 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:35:51.816 INFO:teuthology.orchestra.run.smithi089.stdout:generic/070 22s 2023-07-06T02:35:51.825 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/070 2023-07-06T02:35:51.825 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:51.827 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:52.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:52.173 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2023-07-06T02:35:52.721 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:52.735 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:52.735 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:52.736 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:52.736 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:53.276 INFO:teuthology.orchestra.run.smithi089.stdout:generic/071 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-07-06T02:35:53.281 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/071 2023-07-06T02:35:53.282 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/071 2023-07-06T02:35:53.282 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:53.335 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:53.336 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2023-07-06T02:35:53.679 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:53.683 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:53.684 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:53.685 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:54.283 INFO:teuthology.orchestra.run.smithi089.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-07-06T02:35:54.288 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/072 2023-07-06T02:35:54.288 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/072 2023-07-06T02:35:54.288 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:54.290 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:54.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:54.331 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2023-07-06T02:35:54.675 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:54.680 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:54.680 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:54.681 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:54.682 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:55.193 INFO:teuthology.orchestra.run.smithi089.stdout:generic/073 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:35:55.197 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/073 2023-07-06T02:35:55.198 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/073 2023-07-06T02:35:55.198 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:35:55.203 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:35:55.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:35:55.235 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2023-07-06T02:35:55.595 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:35:55.606 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:35:55.607 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:35:55.608 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:35:55.608 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:36:04.451 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:36:04.454 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:36:05.535 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-07-06T02:36:05.532+0000 7f27b0af0700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 239374) UID: 0 2023-07-06T02:36:05.536 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-07-06T02:36:05.532+0000 7f987d75e700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 239374) UID: 0 2023-07-06T02:36:15.697 INFO:teuthology.orchestra.run.smithi089.stdout:generic/074 20s 2023-07-06T02:36:15.753 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/074 2023-07-06T02:36:15.754 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:36:15.756 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:36:16.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:36:16.560 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2023-07-06T02:36:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:36:18.967 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:36:18.967 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:36:18.970 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:36:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:36:56.649 INFO:teuthology.orchestra.run.smithi089.stdout:generic/075 37s 2023-07-06T02:36:56.650 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/075 2023-07-06T02:36:56.650 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:36:56.650 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:36:56.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:36:56.997 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2023-07-06T02:36:58.081 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:36:58.090 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:36:58.090 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:36:58.091 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:36:58.092 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:36:58.606 INFO:teuthology.orchestra.run.smithi089.stdout:generic/076 [not run] require 172.21.15.89:6789:/scratch to be local device 2023-07-06T02:36:58.611 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/076 2023-07-06T02:36:58.611 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/076 2023-07-06T02:36:58.612 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:36:58.614 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:36:58.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:36:58.663 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2023-07-06T02:36:59.086 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:36:59.092 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:36:59.095 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:36:59.098 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:36:59.098 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:15.537 INFO:teuthology.orchestra.run.smithi089.stdout:generic/077 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-06T02:37:15.543 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/077 2023-07-06T02:37:15.543 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/077 2023-07-06T02:37:15.544 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:37:15.600 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:15.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:37:15.955 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2023-07-06T02:37:16.690 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:37:16.697 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:37:16.698 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:37:16.699 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:37:16.699 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:17.377 INFO:teuthology.orchestra.run.smithi089.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2023-07-06T02:37:17.383 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/078 2023-07-06T02:37:17.383 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/078 2023-07-06T02:37:17.383 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:37:17.391 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:17.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:37:17.552 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2023-07-06T02:37:18.215 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:37:18.224 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:37:18.225 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:37:18.229 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:37:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:18.888 INFO:teuthology.orchestra.run.smithi089.stdout:generic/079 [not run] file system doesn't support chattr +ia 2023-07-06T02:37:18.892 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/079 2023-07-06T02:37:18.892 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/079 2023-07-06T02:37:18.893 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:37:18.894 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:18.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:37:18.930 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2023-07-06T02:37:19.443 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:37:19.448 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:37:19.449 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:37:19.450 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:37:19.450 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:22.000 INFO:teuthology.orchestra.run.smithi089.stdout:generic/080 2s 2023-07-06T02:37:22.217 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/080 2023-07-06T02:37:22.217 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:37:22.224 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:22.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:37:22.252 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2023-07-06T02:37:22.759 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:37:22.778 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:37:22.779 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:37:22.779 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:37:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:23.423 INFO:teuthology.orchestra.run.smithi089.stdout:generic/081 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:37:23.427 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/081 2023-07-06T02:37:23.427 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/081 2023-07-06T02:37:23.427 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:37:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:23.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:37:23.467 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2023-07-06T02:37:23.935 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:37:23.939 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:37:23.939 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:37:23.940 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:37:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:24.440 INFO:teuthology.orchestra.run.smithi089.stdout:generic/082 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-06T02:37:24.447 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/082 2023-07-06T02:37:24.448 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/082 2023-07-06T02:37:24.448 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:37:24.450 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:24.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:37:24.485 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2023-07-06T02:37:24.929 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:37:24.956 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:37:24.956 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:37:24.957 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:37:24.958 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:25.525 INFO:teuthology.orchestra.run.smithi089.stdout:generic/083 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-06T02:37:25.531 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/083 2023-07-06T02:37:25.532 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/083 2023-07-06T02:37:25.532 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:37:25.535 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:25.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:37:25.568 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2023-07-06T02:37:26.116 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:37:26.133 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:37:26.134 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:37:26.134 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:37:26.135 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:37:30.850 INFO:tasks.ceph.mds.c.smithi089.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f27a92e1700 time 2023-07-06T02:37:30.847899+0000 2023-07-06T02:37:30.851 INFO:tasks.ceph.mds.c.smithi089.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-07-06T02:37:30.962 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:30.962 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f27baba5dbf] 2023-07-06T02:37:30.962 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:30.962 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:30.963 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:30.963 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:30.963 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:30.963 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:30.963 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:30.963 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:30.963 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:30.964 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:30.964 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:30.964 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:30.964 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:30.964 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:30.964 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: clone() 2023-07-06T02:37:30.964 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-07-06T02:37:30.959+0000 7f27a92e1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f27a92e1700 time 2023-07-06T02:37:30.847899+0000 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f27baba5dbf] 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:30.965 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:30.966 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:30.966 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:30.966 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:30.966 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:30.966 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:30.966 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:30.966 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:30.967 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:30.967 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:30.967 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:30.967 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: clone() 2023-07-06T02:37:30.967 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:30.967 INFO:tasks.ceph.mds.c.smithi089.stderr:*** Caught signal (Aborted) ** 2023-07-06T02:37:30.967 INFO:tasks.ceph.mds.c.smithi089.stderr: in thread 7f27a92e1700 thread_name:MR_Finisher 2023-07-06T02:37:30.968 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:30.968 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f27b9958ce0] 2023-07-06T02:37:30.968 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: gsignal() 2023-07-06T02:37:30.968 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: abort() 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f27baba5e19] 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:30.969 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 17: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 18: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:30.970 INFO:tasks.ceph.mds.c.smithi089.stderr: 19: clone() 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-07-06T02:37:30.961+0000 7f27a92e1700 -1 *** Caught signal (Aborted) ** 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr: in thread 7f27a92e1700 thread_name:MR_Finisher 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f27b9958ce0] 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: gsignal() 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: abort() 2023-07-06T02:37:30.971 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f27baba5e19] 2023-07-06T02:37:30.972 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:30.972 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:30.972 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:30.972 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:30.972 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:30.972 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:30.972 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 17: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 18: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:30.973 INFO:tasks.ceph.mds.c.smithi089.stderr: 19: clone() 2023-07-06T02:37:30.974 INFO:tasks.ceph.mds.c.smithi089.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-06T02:37:30.974 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:30.990 INFO:tasks.ceph.mds.c.smithi089.stderr: -10> 2023-07-06T02:37:30.959+0000 7f27a92e1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f27a92e1700 time 2023-07-06T02:37:30.847899+0000 2023-07-06T02:37:30.991 INFO:tasks.ceph.mds.c.smithi089.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-07-06T02:37:30.991 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:30.991 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:30.991 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f27baba5dbf] 2023-07-06T02:37:30.991 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:30.992 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:30.993 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:30.993 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:30.993 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:30.993 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:30.993 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:30.993 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: clone() 2023-07-06T02:37:30.993 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:30.994 INFO:tasks.ceph.mds.c.smithi089.stderr: 0> 2023-07-06T02:37:30.961+0000 7f27a92e1700 -1 *** Caught signal (Aborted) ** 2023-07-06T02:37:30.994 INFO:tasks.ceph.mds.c.smithi089.stderr: in thread 7f27a92e1700 thread_name:MR_Finisher 2023-07-06T02:37:30.994 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:30.994 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:30.994 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f27b9958ce0] 2023-07-06T02:37:30.994 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: gsignal() 2023-07-06T02:37:30.995 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: abort() 2023-07-06T02:37:30.995 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f27baba5e19] 2023-07-06T02:37:30.995 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:30.995 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:30.995 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:30.995 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:30.995 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:30.996 INFO:tasks.ceph.mds.c.smithi089.stderr: 17: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:30.997 INFO:tasks.ceph.mds.c.smithi089.stderr: 18: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:30.997 INFO:tasks.ceph.mds.c.smithi089.stderr: 19: clone() 2023-07-06T02:37:30.997 INFO:tasks.ceph.mds.c.smithi089.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-06T02:37:30.997 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:31.013 INFO:tasks.ceph.mds.c.smithi089.stderr: -48> 2023-07-06T02:37:30.959+0000 7f27a92e1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f27a92e1700 time 2023-07-06T02:37:30.847899+0000 2023-07-06T02:37:31.014 INFO:tasks.ceph.mds.c.smithi089.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-07-06T02:37:31.014 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:31.014 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:31.014 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f27baba5dbf] 2023-07-06T02:37:31.014 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:31.014 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:31.015 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:31.015 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:31.015 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:31.015 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:31.015 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:31.015 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:31.015 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: clone() 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:31.016 INFO:tasks.ceph.mds.c.smithi089.stderr: -38> 2023-07-06T02:37:30.961+0000 7f27a92e1700 -1 *** Caught signal (Aborted) ** 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: in thread 7f27a92e1700 thread_name:MR_Finisher 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f27b9958ce0] 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: 2: gsignal() 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: 3: abort() 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f27baba5e19] 2023-07-06T02:37:31.017 INFO:tasks.ceph.mds.c.smithi089.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f27baba5f85] 2023-07-06T02:37:31.018 INFO:tasks.ceph.mds.c.smithi089.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55b603760d92] 2023-07-06T02:37:31.018 INFO:tasks.ceph.mds.c.smithi089.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x55b60374902c] 2023-07-06T02:37:31.018 INFO:tasks.ceph.mds.c.smithi089.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x55b603751c07] 2023-07-06T02:37:31.018 INFO:tasks.ceph.mds.c.smithi089.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x55b60375647f] 2023-07-06T02:37:31.018 INFO:tasks.ceph.mds.c.smithi089.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x55b6035a82c7] 2023-07-06T02:37:31.018 INFO:tasks.ceph.mds.c.smithi089.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x55b6035a9170] 2023-07-06T02:37:31.018 INFO:tasks.ceph.mds.c.smithi089.stderr: 12: (Server::_link_local_finish(boost::intrusive_ptr&, CDentry*, CInode*, unsigned long, unsigned long, bool)+0xf5) [0x55b6035ab075] 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: 13: (C_MDS_link_local_finish::finish(int)+0x35) [0x55b603631745] 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: 14: (MDSContext::complete(int)+0x5f) [0x55b60388fb4f] 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: 15: (MDSIOContextBase::complete(int)+0x524) [0x55b6038902d4] 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: 16: (MDSLogContextBase::complete(int)+0x5b) [0x55b60389068b] 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: 17: (Finisher::finisher_thread_entry()+0x18d) [0x7f27bac498ad] 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: 18: /lib64/libpthread.so.0(+0x81cf) [0x7f27b994e1cf] 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: 19: clone() 2023-07-06T02:37:31.019 INFO:tasks.ceph.mds.c.smithi089.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-06T02:37:31.020 INFO:tasks.ceph.mds.c.smithi089.stderr: 2023-07-06T02:37:35.216 INFO:tasks.ceph.mds.c.smithi089.stderr:daemon-helper: command crashed with signal 6 2023-07-06T02:37:40.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-07-06T02:37:46.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-07-06T02:37:53.336 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2023-07-06T02:37:59.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2023-07-06T02:38:03.303 INFO:teuthology.orchestra.run.smithi089.stdout:generic/084 37s 2023-07-06T02:38:03.714 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/084 2023-07-06T02:38:03.715 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:38:03.717 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:03.729 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:38:03.729 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2023-07-06T02:38:04.546 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-07-06T02:38:04.544+0000 7f9879756700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T02:38:04.707 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:38:04.715 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:38:04.716 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:38:04.717 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:38:04.717 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:05.934 INFO:teuthology.orchestra.run.smithi089.stdout:generic/085 [not run] require 172.21.15.89:6789:/scratch to be valid block disk 2023-07-06T02:38:05.940 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/085 2023-07-06T02:38:05.940 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/085 2023-07-06T02:38:05.940 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:38:05.943 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:05.983 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:38:05.983 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2023-07-06T02:38:06.418 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:38:06.435 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:38:06.436 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:38:06.436 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:38:06.436 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:06.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~26s 2023-07-06T02:38:07.009 INFO:teuthology.orchestra.run.smithi089.stdout:generic/086 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-06T02:38:07.014 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/086 2023-07-06T02:38:07.015 INFO:teuthology.orchestra.run.smithi089.stdout:Not run: generic/086 2023-07-06T02:38:07.015 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:38:07.016 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:07.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:38:07.066 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2023-07-06T02:38:07.562 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:38:07.573 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:38:07.573 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:38:07.574 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:38:07.574 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:08.450 INFO:teuthology.orchestra.run.smithi089.stdout:generic/087 1s 2023-07-06T02:38:08.549 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/087 2023-07-06T02:38:08.549 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:38:08.551 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:08.754 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:38:08.755 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2023-07-06T02:38:09.518 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:38:09.523 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:38:09.524 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:38:09.525 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:38:09.525 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:10.156 INFO:teuthology.orchestra.run.smithi089.stdout:generic/088 1s 2023-07-06T02:38:10.167 INFO:teuthology.orchestra.run.smithi089.stdout:Ran: generic/088 2023-07-06T02:38:10.167 INFO:teuthology.orchestra.run.smithi089.stdout:Passed all 1 tests 2023-07-06T02:38:10.172 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:10.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-06T02:38:10.674 DEBUG:teuthology.orchestra.run.smithi089:> (cd /tmp/tmp.KkaxlHRTc5xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2023-07-06T02:38:11.188 INFO:teuthology.orchestra.run.smithi089.stdout:FSTYP -- ceph 2023-07-06T02:38:11.200 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM -- Linux/x86_64 smithi089 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-06T02:38:11.201 INFO:teuthology.orchestra.run.smithi089.stdout:MKFS_OPTIONS -- 172.21.15.89:6789:/scratch 2023-07-06T02:38:11.202 INFO:teuthology.orchestra.run.smithi089.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBIJaZk9AAeChAA+rDy7GSHggR8nlaNZ5XsOg== -o context=system_u:object_r:root_t:s0 172.21.15.89:6789:/scratch /tmp/tmp.vhpd3PlzjEscratch 2023-07-06T02:38:11.202 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:38:13.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~33s 2023-07-06T02:38:19.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~40s 2023-07-06T02:38:26.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~46s 2023-07-06T02:38:33.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~53s 2023-07-06T02:38:39.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~60s 2023-07-06T02:38:46.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~66s 2023-07-06T02:38:52.839 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~73s 2023-07-06T02:38:59.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~79s 2023-07-06T02:39:06.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~86s 2023-07-06T02:39:12.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~93s 2023-07-06T02:39:16.681 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:39:16.684 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:39:19.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~99s 2023-07-06T02:39:25.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~106s 2023-07-06T02:39:32.502 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~112s 2023-07-06T02:39:39.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~119s 2023-07-06T02:39:45.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~126s 2023-07-06T02:39:47.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:39:47.313 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:39:52.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~132s 2023-07-06T02:39:58.946 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~139s 2023-07-06T02:40:05.556 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~145s 2023-07-06T02:40:12.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~152s 2023-07-06T02:40:17.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:40:17.386 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:40:18.778 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~159s 2023-07-06T02:40:25.389 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~165s 2023-07-06T02:40:32.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~172s 2023-07-06T02:40:38.613 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~178s 2023-07-06T02:40:45.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~185s 2023-07-06T02:40:47.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:40:47.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:40:51.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~192s 2023-07-06T02:40:58.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~198s 2023-07-06T02:41:05.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~205s 2023-07-06T02:41:11.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~212s 2023-07-06T02:41:17.906 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:41:17.908 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:41:18.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~218s 2023-07-06T02:41:24.890 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~225s 2023-07-06T02:41:31.502 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~231s 2023-07-06T02:41:38.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~238s 2023-07-06T02:41:44.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~245s 2023-07-06T02:41:48.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:41:48.217 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:41:51.336 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~251s 2023-07-06T02:41:57.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~258s 2023-07-06T02:42:04.558 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~264s 2023-07-06T02:42:11.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~271s 2023-07-06T02:42:17.781 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~278s 2023-07-06T02:42:18.295 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:42:18.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:42:18.410 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-07-06T02:42:18.407+0000 7f987d75e700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 248495) UID: 0 2023-07-06T02:42:24.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~284s 2023-07-06T02:42:31.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~291s 2023-07-06T02:42:37.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~297s 2023-07-06T02:42:44.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~304s 2023-07-06T02:42:44.223 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-07-06T02:42:44.223 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:42:44.223 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:42:44.841 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:42:44.842 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:42:44.899 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T02:42:45.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.385+0000 7f25c31ff700 1 Processor -- start 2023-07-06T02:42:45.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.385+0000 7f25c31ff700 1 -- start start 2023-07-06T02:42:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.386+0000 7f25c31ff700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25bc0b1490 0x7f25bc0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.386+0000 7f25c31ff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.386+0000 7f25c31ff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25bc06b220 0x7f25bc15de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.386+0000 7f25c31ff700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc0b1da0 con 0x7f25bc0b1490 2023-07-06T02:42:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.386+0000 7f25c31ff700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc06a990 con 0x7f25bc06b220 2023-07-06T02:42:45.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.386+0000 7f25c31ff700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc06ac70 con 0x7f25bc0cc000 2023-07-06T02:42:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25bc0b1490 0x7f25bc0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25bc0b1490 0x7f25bc0caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36608/0 (socket says 172.21.15.89:36608) 2023-07-06T02:42:45.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 -- 172.21.15.89:0/1258211217 learned_addr learned my addr 172.21.15.89:0/1258211217 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:45.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 -- 172.21.15.89:0/1258211217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 msgr2=0x7f25bc0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:45.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25bbfff700 1 --2- 172.21.15.89:0/1258211217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c179c700 1 --2- 172.21.15.89:0/1258211217 >> v2:172.21.15.172:3300/0 conn(0x7f25bc06b220 0x7f25bc15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 --2- 172.21.15.89:0/1258211217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 -- 172.21.15.89:0/1258211217 >> v2:172.21.15.172:3300/0 conn(0x7f25bc06b220 msgr2=0x7f25bc15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 --2- 172.21.15.89:0/1258211217 >> v2:172.21.15.172:3300/0 conn(0x7f25bc06b220 0x7f25bc15de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 -- 172.21.15.89:0/1258211217 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc06d170 con 0x7f25bc0b1490 2023-07-06T02:42:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25c0f9b700 1 --2- 172.21.15.89:0/1258211217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc0b1490 0x7f25bc0caeb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0717a0 tx=0x7f25b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb506d94ad566def server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25bb7fe700 1 -- 172.21.15.89:0/1258211217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b000b870 con 0x7f25bc0b1490 2023-07-06T02:42:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25bb7fe700 1 -- 172.21.15.89:0/1258211217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b000b9d0 con 0x7f25bc0b1490 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.387+0000 7f25bb7fe700 1 -- 172.21.15.89:0/1258211217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b000bd60 con 0x7f25bc0b1490 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 -- 172.21.15.89:0/1258211217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc0b1490 msgr2=0x7f25bc0caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 --2- 172.21.15.89:0/1258211217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc0b1490 0x7f25bc0caeb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0717a0 tx=0x7f25b0009bd0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 -- 172.21.15.89:0/1258211217 shutdown_connections 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 --2- 172.21.15.89:0/1258211217 >> v2:172.21.15.172:3300/0 conn(0x7f25bc06b220 0x7f25bc15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 --2- 172.21.15.89:0/1258211217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc0b1490 0x7f25bc0caeb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 --2- 172.21.15.89:0/1258211217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 -- 172.21.15.89:0/1258211217 >> 172.21.15.89:0/1258211217 conn(0x7f25bc0cc490 msgr2=0x7f25bc0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 -- 172.21.15.89:0/1258211217 shutdown_connections 2023-07-06T02:42:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.388+0000 7f25c31ff700 1 -- 172.21.15.89:0/1258211217 wait complete. 2023-07-06T02:42:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.390+0000 7f25c31ff700 1 Processor -- start 2023-07-06T02:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.390+0000 7f25c31ff700 1 -- start start 2023-07-06T02:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c31ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc06b220 0x7f25bc0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c31ff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc0b1490 0x7f25bc15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c31ff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c31ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc15bd90 con 0x7f25bc06b220 2023-07-06T02:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c31ff700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc15bfc0 con 0x7f25bc0b1490 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c31ff700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc15c2a0 con 0x7f25bc0cc000 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc06b220 0x7f25bc0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc06b220 0x7f25bc0c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36614/0 (socket says 172.21.15.89:36614) 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 -- 172.21.15.89:0/3696791341 learned_addr learned my addr 172.21.15.89:0/3696791341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25bbfff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc0b1490 0x7f25bc15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c179c700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 -- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 msgr2=0x7f25bc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 -- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc0b1490 msgr2=0x7f25bc15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc0b1490 0x7f25bc15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.391+0000 7f25c0f9b700 1 -- 172.21.15.89:0/3696791341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0009550 con 0x7f25bc06b220 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.392+0000 7f25c0f9b700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc06b220 0x7f25bc0c96f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f25bc070270 tx=0x7f25b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.392+0000 7f25b97fa700 1 -- 172.21.15.89:0/3696791341 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0004310 con 0x7f25bc06b220 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.392+0000 7f25b97fa700 1 -- 172.21.15.89:0/3696791341 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b001f070 con 0x7f25bc06b220 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.392+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc15c580 con 0x7f25bc06b220 2023-07-06T02:42:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.392+0000 7f25b97fa700 1 -- 172.21.15.89:0/3696791341 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00036a0 con 0x7f25bc06b220 2023-07-06T02:42:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.392+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc15cb40 con 0x7f25bc06b220 2023-07-06T02:42:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.393+0000 7f25b97fa700 1 -- 172.21.15.89:0/3696791341 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25b0021030 con 0x7f25bc06b220 2023-07-06T02:42:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.393+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c0052f0 con 0x7f25bc06b220 2023-07-06T02:42:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.394+0000 7f25b97fa700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f25a006ace0 0x7f25a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.394+0000 7f25b97fa700 1 -- 172.21.15.89:0/3696791341 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f25b008a960 con 0x7f25bc06b220 2023-07-06T02:42:45.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.394+0000 7f25bbfff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f25a006ace0 0x7f25a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.394+0000 7f25bbfff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f25a006ace0 0x7f25a006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0b0370 tx=0x7f25a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:45.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.399+0000 7f25b97fa700 1 -- 172.21.15.89:0/3696791341 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f25b00558d0 con 0x7f25bc06b220 2023-07-06T02:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.734+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f259c005870 con 0x7f25bc06b220 2023-07-06T02:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.736+0000 7f25b97fa700 1 -- 172.21.15.89:0/3696791341 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18649 (secure 0 0 0) 0x7f25b0058ef0 con 0x7f25bc06b220 2023-07-06T02:42:45.740 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-07-06T02:42:45.740 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":41,"fsid":"b61ed025-dcde-49f5-a87d-8904066cfa54","created":"2023-07-06T02:22:34.414024+0000","modified":"2023-07-06T02:37:49.472752+0000","last_up_change":"2023-07-06T02:22:45.218688+0000","last_in_change":"2023-07-06T02:22:36.854292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T02:22:45.653499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T02:22:46.019299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-06T02:24:08.103741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-06T02:24:08.871993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b518e38-9c91-4acb-80d4-af070fb92b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6801","nonce":990084407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6803","nonce":990084407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6807","nonce":990084407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990084407},{"type":"v1","addr":"172.21.15.89:6805","nonce":990084407}]},"public_addr":"172.21.15.89:6801/990084407","cluster_addr":"172.21.15.89:6803/990084407","heartbeat_back_addr":"172.21.15.89:6807/990084407","heartbeat_front_addr":"172.21.15.89:6805/990084407","state":["exists","up"]},{"osd":1,"uuid":"f4e25425-59b1-484b-b38b-dd3b494a8756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6817","nonce":3305610859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6819","nonce":3305610859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6825","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6827","nonce":3305610859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6821","nonce":3305610859},{"type":"v1","addr":"172.21.15.89:6823","nonce":3305610859}]},"public_addr":"172.21.15.89:6817/3305610859","cluster_addr":"172.21.15.89:6819/3305610859","heartbeat_back_addr":"172.21.15.89:6827/3305610859","heartbeat_front_addr":"172.21.15.89:6823/3305610859","state":["exists","up"]},{"osd":2,"uuid":"c0faa4f3-24e2-4a22-8c3f-e4c021271ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6822","nonce":1124333878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6826","nonce":1124333878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6831","nonce":1124333878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1124333878},{"type":"v1","addr":"172.21.15.89:6829","nonce":1124333878}]},"public_addr":"172.21.15.89:6822/1124333878","cluster_addr":"172.21.15.89:6826/1124333878","heartbeat_back_addr":"172.21.15.89:6831/1124333878","heartbeat_front_addr":"172.21.15.89:6829/1124333878","state":["exists","up"]},{"osd":3,"uuid":"b1e33d2f-29d4-4d0e-b89e-01382e19c83f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6809","nonce":3689514906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6811","nonce":3689514906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6815","nonce":3689514906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3689514906},{"type":"v1","addr":"172.21.15.89:6813","nonce":3689514906}]},"public_addr":"172.21.15.89:6809/3689514906","cluster_addr":"172.21.15.89:6811/3689514906","heartbeat_back_addr":"172.21.15.89:6815/3689514906","heartbeat_front_addr":"172.21.15.89:6813/3689514906","state":["exists","up"]},{"osd":4,"uuid":"5a9e51c3-f007-45af-90c6-0925a630e939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6801","nonce":1682295071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6803","nonce":1682295071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6807","nonce":1682295071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1682295071},{"type":"v1","addr":"172.21.15.172:6805","nonce":1682295071}]},"public_addr":"172.21.15.172:6801/1682295071","cluster_addr":"172.21.15.172:6803/1682295071","heartbeat_back_addr":"172.21.15.172:6807/1682295071","heartbeat_front_addr":"172.21.15.172:6805/1682295071","state":["exists","up"]},{"osd":5,"uuid":"e689f34a-8b34-4de0-a752-ae331230a4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6809","nonce":2248870656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6811","nonce":2248870656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6815","nonce":2248870656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2248870656},{"type":"v1","addr":"172.21.15.172:6813","nonce":2248870656}]},"public_addr":"172.21.15.172:6809/2248870656","cluster_addr":"172.21.15.172:6811/2248870656","heartbeat_back_addr":"172.21.15.172:6815/2248870656","heartbeat_front_addr":"172.21.15.172:6813/2248870656","state":["exists","up"]},{"osd":6,"uuid":"664b96bb-7c04-4a18-b0ce-d96f5b0fffb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6822","nonce":466302387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6826","nonce":466302387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6831","nonce":466302387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":466302387},{"type":"v1","addr":"172.21.15.172:6829","nonce":466302387}]},"public_addr":"172.21.15.172:6822/466302387","cluster_addr":"172.21.15.172:6826/466302387","heartbeat_back_addr":"172.21.15.172:6831/466302387","heartbeat_front_addr":"172.21.15.172:6829/466302387","state":["exists","up"]},{"osd":7,"uuid":"fddd98f5-4991-4a63-9400-64b4c37dc8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6817","nonce":3226277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6819","nonce":3226277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6827","nonce":3226277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":3226277044},{"type":"v1","addr":"172.21.15.172:6823","nonce":3226277044}]},"public_addr":"172.21.15.172:6817/3226277044","cluster_addr":"172.21.15.172:6819/3226277044","heartbeat_back_addr":"172.21.15.172:6827/3226277044","heartbeat_front_addr":"172.21.15.172:6823/3226277044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:41.756947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.118810+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.074181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.020099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.096753+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.126334+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.091083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T02:22:42.145027+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.32","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6837/326075892":"2023-07-07T02:37:49.472693+0000","172.21.15.89:6835/326075892":"2023-07-07T02:37:49.472693+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T02:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f25a006ace0 msgr2=0x7f25a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f25a006ace0 0x7f25a006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0b0370 tx=0x7f25a8008040 comp rx=0 tx=0).stop 2023-07-06T02:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc06b220 msgr2=0x7f25bc0c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc06b220 0x7f25bc0c96f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f25bc070270 tx=0x7f25b0004750 comp rx=0 tx=0).stop 2023-07-06T02:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 shutdown_connections 2023-07-06T02:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25bc0cc000 0x7f25bc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f25a006ace0 0x7f25a006d190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25bc06b220 0x7f25bc0c96f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 --2- 172.21.15.89:0/3696791341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25bc0b1490 0x7f25bc15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 >> 172.21.15.89:0/3696791341 conn(0x7f25bc0cc490 msgr2=0x7f25bc0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 shutdown_connections 2023-07-06T02:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.739+0000 7f25c31ff700 1 -- 172.21.15.89:0/3696791341 wait complete. 2023-07-06T02:42:45.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T02:22:45.653499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T02:22:46.019299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T02:24:08.103741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T02:24:08.871993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T02:42:45.756 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T02:42:45.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.895+0000 7f0bb288f700 1 Processor -- start 2023-07-06T02:42:45.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.895+0000 7f0bb288f700 1 -- start start 2023-07-06T02:42:45.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bb288f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac0c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bb288f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0bac0712f0 0x7f0bac06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bb288f700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0bac06e790 0x7f0bac06b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bb288f700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac0c9fa0 con 0x7f0bac06e790 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bb288f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac06f290 con 0x7f0bac0712f0 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bb288f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac06ecd0 con 0x7f0bac070f20 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0babfff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac0c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bab7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0bac0712f0 0x7f0bac06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bab7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0bac0712f0 0x7f0bac06e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.89:35384/0 (socket says 172.21.15.89:35384) 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0babfff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac0c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.89:59830/0 (socket says 172.21.15.89:59830) 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bab7fe700 1 -- 172.21.15.89:0/3742325667 learned_addr learned my addr 172.21.15.89:0/3742325667 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.896+0000 7f0bb0e2c700 1 --2- 172.21.15.89:0/3742325667 >> v2:172.21.15.89:3300/0 conn(0x7f0bac06e790 0x7f0bac06b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0bb0e2c700 1 -- 172.21.15.89:0/3742325667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 msgr2=0x7f0bac0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0bb0e2c700 1 --2- 172.21.15.89:0/3742325667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac0c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0bb0e2c700 1 -- 172.21.15.89:0/3742325667 >> v2:172.21.15.172:3300/0 conn(0x7f0bac0712f0 msgr2=0x7f0bac06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0bb0e2c700 1 --2- 172.21.15.89:0/3742325667 >> v2:172.21.15.172:3300/0 conn(0x7f0bac0712f0 0x7f0bac06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0bb0e2c700 1 -- 172.21.15.89:0/3742325667 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac06efb0 con 0x7f0bac06e790 2023-07-06T02:42:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0bab7fe700 1 --2- 172.21.15.89:0/3742325667 >> v2:172.21.15.172:3300/0 conn(0x7f0bac0712f0 0x7f0bac06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:42:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0bb0e2c700 1 --2- 172.21.15.89:0/3742325667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac06e790 0x7f0bac06b190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06fad0 tx=0x7f0ba400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20d9e63f000bf54 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:45.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0baaffd700 1 -- 172.21.15.89:0/3742325667 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400e070 con 0x7f0bac06e790 2023-07-06T02:42:45.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0baaffd700 1 -- 172.21.15.89:0/3742325667 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba400bb20 con 0x7f0bac06e790 2023-07-06T02:42:45.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.897+0000 7f0baaffd700 1 -- 172.21.15.89:0/3742325667 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4012430 con 0x7f0bac06e790 2023-07-06T02:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 -- 172.21.15.89:0/3742325667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac06e790 msgr2=0x7f0bac06b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 --2- 172.21.15.89:0/3742325667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac06e790 0x7f0bac06b190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06fad0 tx=0x7f0ba400b330 comp rx=0 tx=0).stop 2023-07-06T02:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 -- 172.21.15.89:0/3742325667 shutdown_connections 2023-07-06T02:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 --2- 172.21.15.89:0/3742325667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac0c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 --2- 172.21.15.89:0/3742325667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac06e790 0x7f0bac06b190 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 --2- 172.21.15.89:0/3742325667 >> v2:172.21.15.172:3300/0 conn(0x7f0bac0712f0 0x7f0bac06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 -- 172.21.15.89:0/3742325667 >> 172.21.15.89:0/3742325667 conn(0x7f0bac0cf1c0 msgr2=0x7f0bac0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 -- 172.21.15.89:0/3742325667 shutdown_connections 2023-07-06T02:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.898+0000 7f0bb288f700 1 -- 172.21.15.89:0/3742325667 wait complete. 2023-07-06T02:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.900+0000 7f0bb288f700 1 Processor -- start 2023-07-06T02:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.900+0000 7f0bb288f700 1 -- start start 2023-07-06T02:42:45.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.901+0000 7f0bb288f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bac06e790 0x7f0bac1f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.901+0000 7f0bb288f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.901+0000 7f0bb288f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac0712f0 0x7f0bac15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.901+0000 7f0bb288f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac158c40 con 0x7f0bac0712f0 2023-07-06T02:42:45.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.901+0000 7f0bb288f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0c8a60 con 0x7f0bac06e790 2023-07-06T02:42:45.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.901+0000 7f0bb288f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1f2610 con 0x7f0bac070f20 2023-07-06T02:42:45.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac0712f0 0x7f0bac15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac0712f0 0x7f0bac15b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36632/0 (socket says 172.21.15.89:36632) 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 -- 172.21.15.89:0/751310770 learned_addr learned my addr 172.21.15.89:0/751310770 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bab7fe700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0babfff700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bac06e790 0x7f0bac1f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 -- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 msgr2=0x7f0bac14c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac14c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 -- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bac06e790 msgr2=0x7f0bac1f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bac06e790 0x7f0bac1f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 -- 172.21.15.89:0/751310770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba000b7f0 con 0x7f0bac0712f0 2023-07-06T02:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb0e2c700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac0712f0 0x7f0bac15b0c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0bac066d50 tx=0x7f0ba4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0ba8ff9700 1 -- 172.21.15.89:0/751310770 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4012430 con 0x7f0bac0712f0 2023-07-06T02:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0ba8ff9700 1 -- 172.21.15.89:0/751310770 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4020070 con 0x7f0bac0712f0 2023-07-06T02:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba4009a70 con 0x7f0bac0712f0 2023-07-06T02:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0ba8ff9700 1 -- 172.21.15.89:0/751310770 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4004280 con 0x7f0bac0712f0 2023-07-06T02:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.902+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac0c8130 con 0x7f0bac0712f0 2023-07-06T02:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.904+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac074270 con 0x7f0bac0712f0 2023-07-06T02:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.904+0000 7f0ba8ff9700 1 -- 172.21.15.89:0/751310770 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ba40043e0 con 0x7f0bac0712f0 2023-07-06T02:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.904+0000 7f0ba8ff9700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0b9806ace0 0x7f0b9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.904+0000 7f0ba8ff9700 1 -- 172.21.15.89:0/751310770 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f0ba4022070 con 0x7f0bac0712f0 2023-07-06T02:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.904+0000 7f0babfff700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0b9806ace0 0x7f0b9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.905+0000 7f0babfff700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0b9806ace0 0x7f0b9806d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009b60 tx=0x7f0b9c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:45.909+0000 7f0ba8ff9700 1 -- 172.21.15.89:0/751310770 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0ba40565a0 con 0x7f0bac0712f0 2023-07-06T02:42:46.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.167+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0bac0ce550 con 0x7f0bac0712f0 2023-07-06T02:42:46.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.168+0000 7f0ba8ff9700 1 -- 172.21.15.89:0/751310770 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0ba4025e90 con 0x7f0bac0712f0 2023-07-06T02:42:46.171 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-07-06T02:42:46.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0b9806ace0 msgr2=0x7f0b9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0b9806ace0 0x7f0b9806d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009b60 tx=0x7f0b9c009210 comp rx=0 tx=0).stop 2023-07-06T02:42:46.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac0712f0 msgr2=0x7f0bac15b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac0712f0 0x7f0bac15b0c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0bac066d50 tx=0x7f0ba4004ad0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 shutdown_connections 2023-07-06T02:42:46.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f0b9806ace0 0x7f0b9806d190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bac0712f0 0x7f0bac15b0c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bac06e790 0x7f0bac1f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 --2- 172.21.15.89:0/751310770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bac070f20 0x7f0bac14c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.170+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 >> 172.21.15.89:0/751310770 conn(0x7f0bac0cf1c0 msgr2=0x7f0bac153480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:46.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.171+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 shutdown_connections 2023-07-06T02:42:46.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.171+0000 7f0bb288f700 1 -- 172.21.15.89:0/751310770 wait complete. 2023-07-06T02:42:46.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T02:42:46.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.315+0000 7f9630d8b700 1 Processor -- start 2023-07-06T02:42:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.315+0000 7f9630d8b700 1 -- start start 2023-07-06T02:42:46.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.316+0000 7f9630d8b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f962c160710 0x7f962c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.316+0000 7f9630d8b700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c0cb9f0 con 0x7f962c15c500 2023-07-06T02:42:46.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.316+0000 7f9630d8b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c15d8e0 con 0x7f962c070f20 2023-07-06T02:42:46.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.316+0000 7f9630d8b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962c071900 con 0x7f962c160710 2023-07-06T02:42:46.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962a59c700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f962c15c500 0x7f962c06edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33306/0 (socket says 172.21.15.89:33306) 2023-07-06T02:42:46.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962a59c700 1 -- 172.21.15.89:0/3217746770 learned_addr learned my addr 172.21.15.89:0/3217746770 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:42:46.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163855032 0 0) 0x7f962c0cb9f0 con 0x7f962c15c500 2023-07-06T02:42:46.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96140035b0 con 0x7f962c15c500 2023-07-06T02:42:46.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962ad9d700 1 --2- 172.21.15.89:0/3217746770 >> v2:172.21.15.172:3300/0 conn(0x7f962c160710 0x7f962c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2082860194 0 0) 0x7f96140035b0 con 0x7f962c15c500 2023-07-06T02:42:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 >> v2:172.21.15.172:3300/0 conn(0x7f962c160710 msgr2=0x7f962c0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 --2- 172.21.15.89:0/3217746770 >> v2:172.21.15.172:3300/0 conn(0x7f962c160710 0x7f962c0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 >> v1:172.21.15.172:6789/0 conn(0x7f962c070f20 legacy=0x7f962c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c160be0 con 0x7f962c15c500 2023-07-06T02:42:46.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1899782731 0 0) 0x7f961c001670 con 0x7f962c15c500 2023-07-06T02:42:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 >> v1:172.21.15.89:6789/0 conn(0x7f962c15c500 legacy=0x7f962c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9614003a00 0x7f9614003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614004850 0x7f9614004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9614005200 0x7f961400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961400c050 con 0x7f9614005200 2023-07-06T02:42:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961400c1c0 con 0x7f9614003a00 2023-07-06T02:42:46.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961400c380 con 0x7f9614004850 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.317+0000 7f962ad9d700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9614005200 0x7f961400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f9629d9b700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614004850 0x7f9614004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962a59c700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9614003a00 0x7f9614003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962ad9d700 1 -- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614004850 msgr2=0x7f9614004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962ad9d700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614004850 0x7f9614004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962ad9d700 1 -- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9614003a00 msgr2=0x7f9614003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962ad9d700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9614003a00 0x7f9614003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962ad9d700 1 -- 172.21.15.89:0/3217746770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140035b0 con 0x7f9614005200 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962ad9d700 1 -- 172.21.15.89:0/3217746770 reap_dead start 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962ad9d700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9614005200 0x7f961400ba10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9624009f90 tx=0x7f9614005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b83032b7e7e8af2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624003a20 con 0x7f9614005200 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962400f040 con 0x7f9614005200 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.318+0000 7f962959a700 1 -- 172.21.15.89:0/3217746770 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624003da0 con 0x7f9614005200 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.319+0000 7f9630d8b700 1 -- 172.21.15.89:0/3217746770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9614005200 msgr2=0x7f961400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.319+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9614005200 0x7f961400ba10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9624009f90 tx=0x7f9614005880 comp rx=0 tx=0).stop 2023-07-06T02:42:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.319+0000 7f9630d8b700 1 -- 172.21.15.89:0/3217746770 shutdown_connections 2023-07-06T02:42:46.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.319+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3217746770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9614005200 0x7f961400ba10 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.319+0000 7f9630d8b700 1 -- 172.21.15.89:0/3217746770 >> 172.21.15.89:0/3217746770 conn(0x7f962c0cf1c0 msgr2=0x7f962c0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:46.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.319+0000 7f9630d8b700 1 -- 172.21.15.89:0/3217746770 shutdown_connections 2023-07-06T02:42:46.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.319+0000 7f9630d8b700 1 -- 172.21.15.89:0/3217746770 wait complete. 2023-07-06T02:42:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.320+0000 7f9630d8b700 1 Processor -- start 2023-07-06T02:42:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.320+0000 7f9630d8b700 1 -- start start 2023-07-06T02:42:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9630d8b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614005200 0x7f962c1e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9630d8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f962c0b2010 0x7f962c0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9630d8b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f962c0aca90 0x7f962c0acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9630d8b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b29c0 con 0x7f962c0b2010 2023-07-06T02:42:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9630d8b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0ad800 con 0x7f962c0aca90 2023-07-06T02:42:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9630d8b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0adae0 con 0x7f9614005200 2023-07-06T02:42:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9629d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f962c0b2010 0x7f962c0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9629d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f962c0b2010 0x7f962c0b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36648/0 (socket says 172.21.15.89:36648) 2023-07-06T02:42:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9629d9b700 1 -- 172.21.15.89:0/3433053743 learned_addr learned my addr 172.21.15.89:0/3433053743 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f962a59c700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614005200 0x7f962c1e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f962ad9d700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f962c0aca90 0x7f962c0acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9629d9b700 1 -- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614005200 msgr2=0x7f962c1e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9629d9b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614005200 0x7f962c1e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9629d9b700 1 -- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f962c0aca90 msgr2=0x7f962c0acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.321+0000 7f9629d9b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f962c0aca90 0x7f962c0acf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.322+0000 7f9629d9b700 1 -- 172.21.15.89:0/3433053743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0addc0 con 0x7f962c0b2010 2023-07-06T02:42:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.322+0000 7f9629d9b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f962c0b2010 0x7f962c0b2480 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f962000f5c0 tx=0x7f962000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.322+0000 7f961affd700 1 -- 172.21.15.89:0/3433053743 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96200012f0 con 0x7f962c0b2010 2023-07-06T02:42:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.322+0000 7f961affd700 1 -- 172.21.15.89:0/3433053743 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96200052e0 con 0x7f962c0b2010 2023-07-06T02:42:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.322+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c0ae0a0 con 0x7f962c0b2010 2023-07-06T02:42:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.322+0000 7f961affd700 1 -- 172.21.15.89:0/3433053743 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620003680 con 0x7f962c0b2010 2023-07-06T02:42:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.322+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c0ad450 con 0x7f962c0b2010 2023-07-06T02:42:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.323+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c066fb0 con 0x7f962c0b2010 2023-07-06T02:42:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.323+0000 7f961affd700 1 -- 172.21.15.89:0/3433053743 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96200037e0 con 0x7f962c0b2010 2023-07-06T02:42:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.324+0000 7f961affd700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f960c06ace0 0x7f960c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.324+0000 7f961affd700 1 -- 172.21.15.89:0/3433053743 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f96200891a0 con 0x7f962c0b2010 2023-07-06T02:42:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.324+0000 7f962a59c700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f960c06ace0 0x7f960c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.325+0000 7f962a59c700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f960c06ace0 0x7f960c06d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f961c0017a0 tx=0x7f961c0015b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.328+0000 7f961affd700 1 -- 172.21.15.89:0/3433053743 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9620054110 con 0x7f962c0b2010 2023-07-06T02:42:46.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.537+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f962c0ae2e0 con 0x7f962c0b2010 2023-07-06T02:42:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.538+0000 7f961affd700 1 -- 172.21.15.89:0/3433053743 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9620057730 con 0x7f962c0b2010 2023-07-06T02:42:46.541 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-07-06T02:42:46.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f960c06ace0 msgr2=0x7f960c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f960c06ace0 0x7f960c06d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f961c0017a0 tx=0x7f961c0015b0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f962c0b2010 msgr2=0x7f962c0b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f962c0b2010 0x7f962c0b2480 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f962000f5c0 tx=0x7f962000f8d0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 shutdown_connections 2023-07-06T02:42:46.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f962c0aca90 0x7f962c0acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9614005200 0x7f962c1e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f960c06ace0 0x7f960c06d190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 --2- 172.21.15.89:0/3433053743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f962c0b2010 0x7f962c0b2480 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 >> 172.21.15.89:0/3433053743 conn(0x7f962c0cf1c0 msgr2=0x7f962c06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:46.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 shutdown_connections 2023-07-06T02:42:46.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.540+0000 7f9630d8b700 1 -- 172.21.15.89:0/3433053743 wait complete. 2023-07-06T02:42:46.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T02:42:46.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.710+0000 7f2397b86700 1 Processor -- start 2023-07-06T02:42:46.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.710+0000 7f2397b86700 1 -- start start 2023-07-06T02:42:46.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.711+0000 7f2397b86700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23900705a0 0x7f23900ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2397b86700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f2390157950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2397b86700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239006e110 con 0x7f23900cd180 2023-07-06T02:42:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2397b86700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239006e3f0 con 0x7f23900705a0 2023-07-06T02:42:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2397b86700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239006e6d0 con 0x7f239006dc10 2023-07-06T02:42:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23900705a0 0x7f23900ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23900705a0 0x7f23900ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36654/0 (socket says 172.21.15.89:36654) 2023-07-06T02:42:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 -- 172.21.15.89:0/3168893287 learned_addr learned my addr 172.21.15.89:0/3168893287 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2396123700 1 --2- 172.21.15.89:0/3168893287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f2390157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 -- 172.21.15.89:0/3168893287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 msgr2=0x7f2390157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:46.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 --2- 172.21.15.89:0/3168893287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f2390157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 -- 172.21.15.89:0/3168893287 >> v1:172.21.15.172:6789/0 conn(0x7f23900cd180 legacy=0x7f239015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 -- 172.21.15.89:0/3168893287 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239006e9b0 con 0x7f23900705a0 2023-07-06T02:42:46.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2395922700 1 --2- 172.21.15.89:0/3168893287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900705a0 0x7f23900ccc40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f23800098f0 tx=0x7f2380009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f737785cd8a4319a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:46.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2394920700 1 -- 172.21.15.89:0/3168893287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380011830 con 0x7f23900705a0 2023-07-06T02:42:46.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2394920700 1 -- 172.21.15.89:0/3168893287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2380011990 con 0x7f23900705a0 2023-07-06T02:42:46.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.712+0000 7f2394920700 1 -- 172.21.15.89:0/3168893287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380011d20 con 0x7f23900705a0 2023-07-06T02:42:46.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 -- 172.21.15.89:0/3168893287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900705a0 msgr2=0x7f23900ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 --2- 172.21.15.89:0/3168893287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900705a0 0x7f23900ccc40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f23800098f0 tx=0x7f2380009c00 comp rx=0 tx=0).stop 2023-07-06T02:42:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 -- 172.21.15.89:0/3168893287 shutdown_connections 2023-07-06T02:42:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 --2- 172.21.15.89:0/3168893287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f2390157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 --2- 172.21.15.89:0/3168893287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900705a0 0x7f23900ccc40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 -- 172.21.15.89:0/3168893287 >> 172.21.15.89:0/3168893287 conn(0x7f23900d0b00 msgr2=0x7f23900d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 -- 172.21.15.89:0/3168893287 shutdown_connections 2023-07-06T02:42:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.713+0000 7f2397b86700 1 -- 172.21.15.89:0/3168893287 wait complete. 2023-07-06T02:42:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.715+0000 7f2397b86700 1 Processor -- start 2023-07-06T02:42:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.715+0000 7f2397b86700 1 -- start start 2023-07-06T02:42:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.716+0000 7f2397b86700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f23901f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.716+0000 7f2397b86700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23900705a0 0x7f23900ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.716+0000 7f2397b86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900cd180 0x7f23900cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.716+0000 7f2397b86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2390147380 con 0x7f23900cd180 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.716+0000 7f2397b86700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23901f32e0 con 0x7f23900705a0 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.716+0000 7f2397b86700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23901f38f0 con 0x7f239006dc10 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900cd180 0x7f23900cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900cd180 0x7f23900cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36664/0 (socket says 172.21.15.89:36664) 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 -- 172.21.15.89:0/1174326509 learned_addr learned my addr 172.21.15.89:0/1174326509 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:46.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2395121700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23900705a0 0x7f23900ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2395922700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f23901f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 -- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 msgr2=0x7f23901f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f23901f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 -- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23900705a0 msgr2=0x7f23900ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23900705a0 0x7f23900ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:46.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 -- 172.21.15.89:0/1174326509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380009550 con 0x7f23900cd180 2023-07-06T02:42:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2396123700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900cd180 0x7f23900cebc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f238800efc0 tx=0x7f238800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f23867fc700 1 -- 172.21.15.89:0/1174326509 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388009ea0 con 0x7f23900cd180 2023-07-06T02:42:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f23867fc700 1 -- 172.21.15.89:0/1174326509 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238800f040 con 0x7f23900cd180 2023-07-06T02:42:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23901f3bd0 con 0x7f23900cd180 2023-07-06T02:42:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f23867fc700 1 -- 172.21.15.89:0/1174326509 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800ced0 con 0x7f23900cd180 2023-07-06T02:42:46.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.717+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23901f4190 con 0x7f23900cd180 2023-07-06T02:42:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.718+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2390074270 con 0x7f23900cd180 2023-07-06T02:42:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.719+0000 7f23867fc700 1 -- 172.21.15.89:0/1174326509 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2388014480 con 0x7f23900cd180 2023-07-06T02:42:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.719+0000 7f23867fc700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f237406ace0 0x7f237406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:46.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.719+0000 7f23867fc700 1 -- 172.21.15.89:0/1174326509 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f2388089e20 con 0x7f23900cd180 2023-07-06T02:42:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.719+0000 7f2395922700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f237406ace0 0x7f237406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:46.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.720+0000 7f2395922700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f237406ace0 0x7f237406d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2380009c40 tx=0x7f2380008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:46.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.725+0000 7f23867fc700 1 -- 172.21.15.89:0/1174326509 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2388054d90 con 0x7f23900cd180 2023-07-06T02:42:46.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.994+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f239006f590 con 0x7f23900cd180 2023-07-06T02:42:46.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.995+0000 7f23867fc700 1 -- 172.21.15.89:0/1174326509 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2388018020 con 0x7f23900cd180 2023-07-06T02:42:46.998 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-07-06T02:42:47.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f237406ace0 msgr2=0x7f237406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f237406ace0 0x7f237406d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2380009c40 tx=0x7f2380008030 comp rx=0 tx=0).stop 2023-07-06T02:42:47.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900cd180 msgr2=0x7f23900cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900cd180 0x7f23900cebc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f238800efc0 tx=0x7f238800c3a0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 shutdown_connections 2023-07-06T02:42:47.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f237406ace0 0x7f237406d190 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23900cd180 0x7f23900cebc0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f239006dc10 0x7f23901f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 --2- 172.21.15.89:0/1174326509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23900705a0 0x7f23900ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 >> 172.21.15.89:0/1174326509 conn(0x7f23900d0b00 msgr2=0x7f239015d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:47.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 shutdown_connections 2023-07-06T02:42:47.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:46.997+0000 7f2397b86700 1 -- 172.21.15.89:0/1174326509 wait complete. 2023-07-06T02:42:47.014 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T02:42:47.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.152+0000 7fdf72a16700 1 Processor -- start 2023-07-06T02:42:47.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.152+0000 7fdf72a16700 1 -- start start 2023-07-06T02:42:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf72a16700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c071390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf72a16700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf6c160000 0x7fdf6c160480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf72a16700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6c0cb9f0 con 0x7fdf6c15c500 2023-07-06T02:42:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf72a16700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6c0718d0 con 0x7fdf6c160000 2023-07-06T02:42:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf72a16700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c1609c0 con 0x7fdf6c070f20 2023-07-06T02:42:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf6bfff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdf6c15c500 0x7fdf6c06edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33338/0 (socket says 172.21.15.89:33338) 2023-07-06T02:42:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf6bfff700 1 -- 172.21.15.89:0/3316677462 learned_addr learned my addr 172.21.15.89:0/3316677462 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf70fb3700 1 --2- 172.21.15.89:0/3316677462 >> v2:172.21.15.89:3300/0 conn(0x7fdf6c160000 0x7fdf6c160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf6b7fe700 1 --2- 172.21.15.89:0/3316677462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf70fb3700 1 -- 172.21.15.89:0/3316677462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 msgr2=0x7fdf6c071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf70fb3700 1 --2- 172.21.15.89:0/3316677462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c071390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf70fb3700 1 -- 172.21.15.89:0/3316677462 >> v1:172.21.15.89:6789/0 conn(0x7fdf6c15c500 legacy=0x7fdf6c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf70fb3700 1 -- 172.21.15.89:0/3316677462 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c15d8e0 con 0x7fdf6c160000 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf6affd700 1 -- 172.21.15.89:0/3316677462 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392628152 0 0) 0x7fdf6c0cb9f0 con 0x7fdf6c15c500 2023-07-06T02:42:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.153+0000 7fdf70fb3700 1 --2- 172.21.15.89:0/3316677462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c160000 0x7fdf6c160480 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009dc0 tx=0x7fdf6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa241f7cca9add9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf6affd700 1 -- 172.21.15.89:0/3316677462 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400e070 con 0x7fdf6c160000 2023-07-06T02:42:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf6affd700 1 -- 172.21.15.89:0/3316677462 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6400bb90 con 0x7fdf6c160000 2023-07-06T02:42:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf6affd700 1 -- 172.21.15.89:0/3316677462 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64012430 con 0x7fdf6c160000 2023-07-06T02:42:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf72a16700 1 -- 172.21.15.89:0/3316677462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c160000 msgr2=0x7fdf6c160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf72a16700 1 --2- 172.21.15.89:0/3316677462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c160000 0x7fdf6c160480 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009dc0 tx=0x7fdf6400b330 comp rx=0 tx=0).stop 2023-07-06T02:42:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf72a16700 1 -- 172.21.15.89:0/3316677462 shutdown_connections 2023-07-06T02:42:47.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf72a16700 1 --2- 172.21.15.89:0/3316677462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c160000 0x7fdf6c160480 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf72a16700 1 --2- 172.21.15.89:0/3316677462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c071390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf72a16700 1 -- 172.21.15.89:0/3316677462 >> 172.21.15.89:0/3316677462 conn(0x7fdf6c0cf1c0 msgr2=0x7fdf6c0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.154+0000 7fdf72a16700 1 -- 172.21.15.89:0/3316677462 shutdown_connections 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.155+0000 7fdf72a16700 1 -- 172.21.15.89:0/3316677462 wait complete. 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.156+0000 7fdf72a16700 1 Processor -- start 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.156+0000 7fdf72a16700 1 -- start start 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf72a16700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf72a16700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c15c500 0x7fdf6c1f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf72a16700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf6c160000 0x7fdf6c06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf72a16700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0c58c0 con 0x7fdf6c15c500 2023-07-06T02:42:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf72a16700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0c4c40 con 0x7fdf6c160000 2023-07-06T02:42:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf72a16700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0c4f20 con 0x7fdf6c070f20 2023-07-06T02:42:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c15c500 0x7fdf6c1f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c15c500 0x7fdf6c1f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36676/0 (socket says 172.21.15.89:36676) 2023-07-06T02:42:47.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6b7fe700 1 -- 172.21.15.89:0/2104225879 learned_addr learned my addr 172.21.15.89:0/2104225879 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6bfff700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf70fb3700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf6c160000 0x7fdf6c06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6bfff700 1 -- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf6c160000 msgr2=0x7fdf6c06aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6bfff700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf6c160000 0x7fdf6c06aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6bfff700 1 -- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c15c500 msgr2=0x7fdf6c1f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6bfff700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c15c500 0x7fdf6c1f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6bfff700 1 -- 172.21.15.89:0/2104225879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64009a70 con 0x7fdf6c070f20 2023-07-06T02:42:47.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6b7fe700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c15c500 0x7fdf6c1f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T02:42:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.157+0000 7fdf6bfff700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c0c4660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c4b0 tx=0x7fdf5c00c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.158+0000 7fdf68ff9700 1 -- 172.21.15.89:0/2104225879 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c004c80 con 0x7fdf6c070f20 2023-07-06T02:42:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.158+0000 7fdf68ff9700 1 -- 172.21.15.89:0/2104225879 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c004e00 con 0x7fdf6c070f20 2023-07-06T02:42:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.158+0000 7fdf68ff9700 1 -- 172.21.15.89:0/2104225879 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c007310 con 0x7fdf6c070f20 2023-07-06T02:42:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.158+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c0c5200 con 0x7fdf6c070f20 2023-07-06T02:42:47.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.158+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c1592e0 con 0x7fdf6c070f20 2023-07-06T02:42:47.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.159+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6c0b1b70 con 0x7fdf6c070f20 2023-07-06T02:42:47.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.160+0000 7fdf68ff9700 1 -- 172.21.15.89:0/2104225879 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf5c007470 con 0x7fdf6c070f20 2023-07-06T02:42:47.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.160+0000 7fdf68ff9700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf5806ace0 0x7fdf5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.160+0000 7fdf68ff9700 1 -- 172.21.15.89:0/2104225879 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6526+0+0 (secure 0 0 0) 0x7fdf5c0891d0 con 0x7fdf6c070f20 2023-07-06T02:42:47.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.160+0000 7fdf6b7fe700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf5806ace0 0x7fdf5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.161+0000 7fdf6b7fe700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf5806ace0 0x7fdf5806d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0c93e0 tx=0x7fdf60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:47.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.165+0000 7fdf68ff9700 1 -- 172.21.15.89:0/2104225879 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf5c054200 con 0x7fdf6c070f20 2023-07-06T02:42:47.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.382+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf6c074270 con 0x7fdf6c070f20 2023-07-06T02:42:47.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.382+0000 7fdf68ff9700 1 -- 172.21.15.89:0/2104225879 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf5c057820 con 0x7fdf6c070f20 2023-07-06T02:42:47.386 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-07-06T02:42:47.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf5806ace0 msgr2=0x7fdf5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf5806ace0 0x7fdf5806d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0c93e0 tx=0x7fdf60008040 comp rx=0 tx=0).stop 2023-07-06T02:42:47.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 msgr2=0x7fdf6c0c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c0c4660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c4b0 tx=0x7fdf5c00c7c0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 shutdown_connections 2023-07-06T02:42:47.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf6c160000 0x7fdf6c06aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf6c070f20 0x7fdf6c0c4660 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7fdf5806ace0 0x7fdf5806d190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 --2- 172.21.15.89:0/2104225879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6c15c500 0x7fdf6c1f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 >> 172.21.15.89:0/2104225879 conn(0x7fdf6c0cf1c0 msgr2=0x7fdf6c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:47.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 shutdown_connections 2023-07-06T02:42:47.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.385+0000 7fdf72a16700 1 -- 172.21.15.89:0/2104225879 wait complete. 2023-07-06T02:42:47.401 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T02:42:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.665+0000 7f35d6fe6700 1 Processor -- start 2023-07-06T02:42:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.665+0000 7f35d6fe6700 1 -- start start 2023-07-06T02:42:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d6fe6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d6fe6700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35d006b190 0x7f35d006b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d6fe6700 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00c9fa0 con 0x7f35d00712f0 2023-07-06T02:42:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d6fe6700 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d006f290 con 0x7f35d006b190 2023-07-06T02:42:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d6fe6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d006ecd0 con 0x7f35d0070f20 2023-07-06T02:42:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35cffff700 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f35d00712f0 0x7f35d006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33354/0 (socket says 172.21.15.89:33354) 2023-07-06T02:42:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d5583700 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35d006b190 0x7f35d006b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35cffff700 1 -- 172.21.15.89:0/2117423001 learned_addr learned my addr 172.21.15.89:0/2117423001 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-07-06T02:42:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d4d82700 1 --2- 172.21.15.89:0/2117423001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d5583700 1 -- 172.21.15.89:0/2117423001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 msgr2=0x7f35d00c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T02:42:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d5583700 1 --2- 172.21.15.89:0/2117423001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d5583700 1 -- 172.21.15.89:0/2117423001 >> v1:172.21.15.89:6789/0 conn(0x7f35d00712f0 legacy=0x7f35d006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d5583700 1 -- 172.21.15.89:0/2117423001 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d006efb0 con 0x7f35d006b190 2023-07-06T02:42:47.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.666+0000 7f35d5583700 1 --2- 172.21.15.89:0/2117423001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d006b190 0x7f35d006b610 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009d90 tx=0x7f35c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8ab66fc7778d72 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:47.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35cf7fe700 1 -- 172.21.15.89:0/2117423001 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c800e070 con 0x7f35d006b190 2023-07-06T02:42:47.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35cf7fe700 1 -- 172.21.15.89:0/2117423001 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c800bb20 con 0x7f35d006b190 2023-07-06T02:42:47.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35cf7fe700 1 -- 172.21.15.89:0/2117423001 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8012430 con 0x7f35d006b190 2023-07-06T02:42:47.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35d6fe6700 1 -- 172.21.15.89:0/2117423001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d006b190 msgr2=0x7f35d006b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/2117423001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d006b190 0x7f35d006b610 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009d90 tx=0x7f35c800b330 comp rx=0 tx=0).stop 2023-07-06T02:42:47.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35d6fe6700 1 -- 172.21.15.89:0/2117423001 shutdown_connections 2023-07-06T02:42:47.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/2117423001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d006b190 0x7f35d006b610 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/2117423001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35d6fe6700 1 -- 172.21.15.89:0/2117423001 >> 172.21.15.89:0/2117423001 conn(0x7f35d00cf1c0 msgr2=0x7f35d00cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:47.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.667+0000 7f35d6fe6700 1 -- 172.21.15.89:0/2117423001 shutdown_connections 2023-07-06T02:42:47.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.668+0000 7f35d6fe6700 1 -- 172.21.15.89:0/2117423001 wait complete. 2023-07-06T02:42:47.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.669+0000 7f35d6fe6700 1 Processor -- start 2023-07-06T02:42:47.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.669+0000 7f35d6fe6700 1 -- start start 2023-07-06T02:42:47.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d6fe6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35d006b190 0x7f35d014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d6fe6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d6fe6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d00712f0 0x7f35d00c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d6fe6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d0145b30 con 0x7f35d00712f0 2023-07-06T02:42:47.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d6fe6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00c7b10 con 0x7f35d006b190 2023-07-06T02:42:47.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d6fe6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00c7df0 con 0x7f35d0070f20 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d5583700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d00712f0 0x7f35d00c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d5583700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d00712f0 0x7f35d00c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36690/0 (socket says 172.21.15.89:36690) 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.670+0000 7f35d5583700 1 -- 172.21.15.89:0/3445727515 learned_addr learned my addr 172.21.15.89:0/3445727515 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d4d82700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35d006b190 0x7f35d014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35cffff700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d5583700 1 -- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 msgr2=0x7f35d00af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d5583700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d5583700 1 -- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35d006b190 msgr2=0x7f35d014a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d5583700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35d006b190 0x7f35d014a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d5583700 1 -- 172.21.15.89:0/3445727515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009a70 con 0x7f35d00712f0 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d5583700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d00712f0 0x7f35d00c7410 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f35c8014040 tx=0x7f35c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35cd7fa700 1 -- 172.21.15.89:0/3445727515 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8012430 con 0x7f35d00712f0 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35cd7fa700 1 -- 172.21.15.89:0/3445727515 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c8020070 con 0x7f35d00712f0 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35cd7fa700 1 -- 172.21.15.89:0/3445727515 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8003ea0 con 0x7f35d00712f0 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d00c80d0 con 0x7f35d00712f0 2023-07-06T02:42:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.671+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d00c87b0 con 0x7f35d00712f0 2023-07-06T02:42:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.673+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d00b32c0 con 0x7f35d00712f0 2023-07-06T02:42:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.673+0000 7f35cd7fa700 1 -- 172.21.15.89:0/3445727515 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35c8004040 con 0x7f35d00712f0 2023-07-06T02:42:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.673+0000 7f35cd7fa700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35b406ace0 0x7f35b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T02:42:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.673+0000 7f35cd7fa700 1 -- 172.21.15.89:0/3445727515 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f35c8022070 con 0x7f35d00712f0 2023-07-06T02:42:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.673+0000 7f35d4d82700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35b406ace0 0x7f35b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T02:42:47.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.674+0000 7f35d4d82700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35b406ace0 0x7f35b406d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35d00c84d0 tx=0x7f35c4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T02:42:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.678+0000 7f35cd7fa700 1 -- 172.21.15.89:0/3445727515 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35d00b32c0 con 0x7f35d00712f0 2023-07-06T02:42:47.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.906+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35d00b32c0 con 0x7f35d00712f0 2023-07-06T02:42:47.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.907+0000 7f35cd7fa700 1 -- 172.21.15.89:0/3445727515 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) v1 ==== 80+0+120 (secure 0 0 0) 0x7f35c802c090 con 0x7f35d00712f0 2023-07-06T02:42:47.910 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6837/326075892 2023-07-07T02:37:49.472693+0000 2023-07-06T02:42:47.910 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6835/326075892 2023-07-07T02:37:49.472693+0000 2023-07-06T02:42:47.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35b406ace0 msgr2=0x7f35b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35b406ace0 0x7f35b406d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35d00c84d0 tx=0x7f35c4009210 comp rx=0 tx=0).stop 2023-07-06T02:42:47.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d00712f0 msgr2=0x7f35d00c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T02:42:47.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d00712f0 0x7f35d00c7410 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f35c8014040 tx=0x7f35c8004ad0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 shutdown_connections 2023-07-06T02:42:47.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:6832/3591115711,v1:172.21.15.89:6833/3591115711] conn(0x7f35b406ace0 0x7f35b406d190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d00712f0 0x7f35d00c7410 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35d006b190 0x7f35d014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 --2- 172.21.15.89:0/3445727515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35d0070f20 0x7f35d00af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T02:42:47.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 >> 172.21.15.89:0/3445727515 conn(0x7f35d00cf1c0 msgr2=0x7f35d00caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T02:42:47.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 shutdown_connections 2023-07-06T02:42:47.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-07-06T02:42:47.910+0000 7f35d6fe6700 1 -- 172.21.15.89:0/3445727515 wait complete. 2023-07-06T02:42:47.915 INFO:teuthology.orchestra.run.smithi089.stderr:listed 2 entries 2023-07-06T02:42:47.927 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-06T02:42:47.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:42:47.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-07-06T02:42:47.967 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:42:47.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:42:47.967 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T02:42:48.061 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-07-06T02:42:48.062 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:42:48.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20494 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 172.21.15.89:6789:/test /tmp/tmp.aPxbH4RFvvtest ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=b61ed025-dcde-49f5-a87d-8904066cfa54,acl 0 0 2023-07-06T02:42:48.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-07-06T02:42:48.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:42:48.079 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T02:42:48.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:42:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T02:42:48.136 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:42:48.136 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:42:48.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12873 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T02:42:48.156 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:42:48.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:42:48.156 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T02:42:48.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:42:48.214 INFO:teuthology.orchestra.run.smithi172.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-07-06T02:42:48.214 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-07-06T02:42:48.215 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T02:42:48.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12873 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T02:42:48.272 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-07-06T02:42:48.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T02:42:48.272 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T02:42:48.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T02:42:48.329 INFO:teuthology.orchestra.run.smithi172.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T02:42:49.938 INFO:tasks.ceph.osd.0:Sent signal 15 2023-07-06T02:42:49.939 INFO:tasks.ceph.osd.1:Sent signal 15 2023-07-06T02:42:49.939 INFO:tasks.ceph.osd.2:Sent signal 15 2023-07-06T02:42:49.939 INFO:tasks.ceph.osd.3:Sent signal 15 2023-07-06T02:42:49.940 INFO:tasks.ceph.osd.4:Sent signal 15 2023-07-06T02:42:49.940 INFO:tasks.ceph.osd.5:Sent signal 15 2023-07-06T02:42:49.940 INFO:tasks.ceph.osd.6:Sent signal 15 2023-07-06T02:42:49.941 INFO:tasks.ceph.osd.7:Sent signal 15 2023-07-06T02:42:49.941 INFO:tasks.ceph.mds.a:Sent signal 15 2023-07-06T02:42:49.941 INFO:tasks.ceph.mds.b:Sent signal 15 2023-07-06T02:42:49.942 INFO:tasks.ceph.mds.d:Sent signal 15 2023-07-06T02:42:49.942 INFO:tasks.ceph.mon.a:Sent signal 15 2023-07-06T02:42:49.942 INFO:tasks.ceph.mon.b:Sent signal 15 2023-07-06T02:42:49.942 INFO:tasks.ceph.mon.c:Sent signal 15 2023-07-06T02:42:49.943 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-07-06T02:42:49.943 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-07-06T02:42:49.944 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-07-06T02:42:49.936+0000 7fc27b56e700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 131292) UID: 0 2023-07-06T02:42:49.944 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-07-06T02:42:49.936+0000 7fc27b56e700 -1 osd.0 41 *** Got signal Terminated *** 2023-07-06T02:42:49.944 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-07-06T02:42:49.936+0000 7fc27b56e700 -1 osd.0 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:49.944 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-07-06T02:42:49.937+0000 7f6b8d040700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 131296) UID: 0 2023-07-06T02:42:49.945 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-07-06T02:42:49.937+0000 7f6b8d040700 -1 osd.1 41 *** Got signal Terminated *** 2023-07-06T02:42:49.945 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-07-06T02:42:49.937+0000 7f6b8d040700 -1 osd.1 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:49.945 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-07-06T02:42:49.937+0000 7f5139095700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 131297) UID: 0 2023-07-06T02:42:49.945 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-07-06T02:42:49.937+0000 7f5139095700 -1 osd.2 41 *** Got signal Terminated *** 2023-07-06T02:42:49.945 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-07-06T02:42:49.937+0000 7f5139095700 -1 osd.2 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:49.950 INFO:tasks.ceph.mon.c.smithi172.stderr:2023-07-06T02:42:49.948+0000 7fb08e5f5700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 130926) UID: 0 2023-07-06T02:42:49.950 INFO:tasks.ceph.mon.c.smithi172.stderr:2023-07-06T02:42:49.948+0000 7fb08e5f5700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-07-06T02:42:49.951 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f155c03b700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 131575) UID: 0 2023-07-06T02:42:49.951 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f155c03b700 -1 osd.4 41 *** Got signal Terminated *** 2023-07-06T02:42:49.951 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f155c03b700 -1 osd.4 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:49.951 INFO:tasks.ceph.mds.d.smithi172.stderr:2023-07-06T02:42:49.948+0000 7ff7423d2700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 139088) UID: 0 2023-07-06T02:42:49.951 INFO:tasks.ceph.mds.d.smithi172.stderr:2023-07-06T02:42:49.948+0000 7ff7423d2700 -1 mds.d *** got signal Terminated *** 2023-07-06T02:42:49.952 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-07-06T02:42:49.938+0000 7f99fb950700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 131300) UID: 0 2023-07-06T02:42:49.952 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-07-06T02:42:49.938+0000 7f99fb950700 -1 osd.3 41 *** Got signal Terminated *** 2023-07-06T02:42:49.952 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-07-06T02:42:49.938+0000 7f99fb950700 -1 osd.3 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:49.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-07-06T02:42:49.948+0000 7f7443e54700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 130971) UID: 0 2023-07-06T02:42:49.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-07-06T02:42:49.948+0000 7f7443e54700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-06T02:42:49.953 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-07-06T02:42:49.948+0000 7f987d75e700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 147955) UID: 0 2023-07-06T02:42:49.953 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-07-06T02:42:49.948+0000 7f987d75e700 -1 mds.a *** got signal Terminated *** 2023-07-06T02:42:49.954 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f4123e2f700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 131578) UID: 0 2023-07-06T02:42:49.954 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f4123e2f700 -1 osd.5 41 *** Got signal Terminated *** 2023-07-06T02:42:49.954 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f4123e2f700 -1 osd.5 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:49.954 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f2ddeb55700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 131583) UID: 0 2023-07-06T02:42:49.954 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f2ddeb55700 -1 osd.7 41 *** Got signal Terminated *** 2023-07-06T02:42:49.955 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f2ddeb55700 -1 osd.7 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:49.956 INFO:tasks.ceph.mds.b.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f0e11d25700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 139087) UID: 0 2023-07-06T02:42:49.956 INFO:tasks.ceph.mds.b.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f0e11d25700 -1 mds.b *** got signal Terminated *** 2023-07-06T02:42:49.956 INFO:tasks.ceph.mon.b.smithi172.stderr:2023-07-06T02:42:49.954+0000 7f0a29a80700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 130924) UID: 0 2023-07-06T02:42:49.957 INFO:tasks.ceph.mon.b.smithi172.stderr:2023-07-06T02:42:49.954+0000 7f0a29a80700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-07-06T02:42:49.997 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f58c4d58700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 131581) UID: 0 2023-07-06T02:42:49.997 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f58c4d58700 -1 osd.6 41 *** Got signal Terminated *** 2023-07-06T02:42:49.997 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-07-06T02:42:49.948+0000 7f58c4d58700 -1 osd.6 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T02:42:50.168 INFO:tasks.ceph.mgr.x.smithi172.stderr:daemon-helper: command crashed with signal 15 2023-07-06T02:44:57.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:44:57.045 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:45:27.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:45:27.122 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:45:57.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:45:57.195 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:46:27.452 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:46:27.455 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:46:57.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:46:57.589 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:47:27.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:47:27.654 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:47:57.805 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:47:57.808 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:48:27.909 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:48:27.912 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:48:57.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:48:57.992 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:49:28.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:49:28.061 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:49:58.187 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:49:58.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:50:28.322 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:50:28.325 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:50:58.395 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:50:58.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:51:28.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:51:28.595 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:51:58.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:51:58.663 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:52:28.724 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:52:28.727 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:52:58.793 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:52:58.796 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:53:28.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:53:28.866 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:53:59.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:53:59.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:54:29.581 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:54:29.584 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:54:59.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:54:59.778 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:55:29.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:55:29.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:56:00.059 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:56:00.062 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:56:30.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:56:30.132 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:57:00.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:57:00.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:57:30.736 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:57:30.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:58:00.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:58:00.818 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:58:30.969 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:58:30.972 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:59:01.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:59:01.058 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:59:31.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T02:59:31.126 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:00:01.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:00:01.275 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:00:31.635 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:00:31.638 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:01:01.710 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:01:01.713 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:01:31.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:01:31.803 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:02:01.888 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:02:01.892 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:02:31.958 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:02:31.961 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:03:02.039 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:03:02.042 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:03:32.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:03:32.336 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:04:02.412 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:04:02.416 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:04:32.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:04:32.493 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:05:02.645 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:05:02.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:05:32.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:05:32.994 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:06:03.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:06:03.115 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:06:33.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:06:33.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:07:03.360 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:07:03.363 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:07:33.498 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:07:33.502 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:08:03.792 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:08:03.795 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:08:33.862 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:08:33.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:09:03.933 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:09:03.936 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:09:34.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:09:34.082 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:10:04.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:10:04.370 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:10:34.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:10:34.495 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:11:04.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:11:04.775 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:11:34.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:11:34.834 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:12:04.961 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:12:04.964 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:12:35.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:12:35.288 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:13:05.355 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:13:05.358 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:13:35.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:13:35.424 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:14:05.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:14:05.772 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:14:35.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:14:35.883 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:15:06.043 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:15:06.046 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:15:36.121 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:15:36.124 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:16:06.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:16:06.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:16:36.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:16:36.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:17:06.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:17:06.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:17:36.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:17:36.689 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:18:06.801 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:18:06.804 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:18:36.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:18:36.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:19:07.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:19:07.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:19:37.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:19:37.307 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:20:07.395 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:20:07.398 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:20:37.471 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:20:37.474 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:21:07.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:21:07.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:21:37.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:21:37.694 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:22:07.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:22:07.836 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:22:37.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:22:37.900 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:23:07.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:23:07.969 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:23:38.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:23:38.073 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:24:08.383 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:24:08.386 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:24:38.461 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:24:38.465 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:25:08.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:25:08.603 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:25:38.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:25:38.947 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:26:09.029 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:26:09.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:26:39.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:26:39.103 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:27:09.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:27:09.192 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:27:39.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:27:39.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:28:09.448 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:28:09.451 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:28:39.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:28:39.519 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:29:09.579 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:29:09.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:29:39.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:29:39.754 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:30:10.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:30:10.057 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:30:40.157 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:30:40.161 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:31:10.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:31:10.249 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:31:40.481 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:31:40.484 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:32:10.539 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:32:10.541 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:32:40.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:32:40.609 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:33:10.710 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:33:10.713 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:33:40.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:33:40.783 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:34:10.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:34:10.847 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:34:40.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:34:40.991 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:35:11.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:35:11.061 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:35:41.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:35:41.140 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:36:11.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:36:11.211 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:36:41.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:36:41.341 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:37:11.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:37:11.408 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:37:41.487 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:37:41.490 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:38:11.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:38:11.618 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:38:41.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:38:41.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:39:11.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:39:11.758 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:39:41.823 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:39:41.826 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:40:12.030 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:40:12.038 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:40:42.351 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:40:42.354 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:41:12.488 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:41:12.491 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:41:42.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:41:42.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:42:12.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:42:12.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:42:42.724 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:42:42.727 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:43:12.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:43:12.901 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:43:42.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:43:42.969 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:13.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:13.031 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:43.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:43.359 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:13.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:13.724 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:43.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:43.834 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:14.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:14.158 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:44.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:44.236 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:14.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:14.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:44.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:44.829 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:14.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:14.962 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:45.026 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:45.030 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:15.101 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:15.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:45.238 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:45.241 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:15.299 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:15.302 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:45.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:45.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:15.515 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:15.519 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:45.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:45.586 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:15.747 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:15.750 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:45.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:45.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:15.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:15.969 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:46.035 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:46.038 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:16.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:16.112 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:46.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:46.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:16.579 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:16.582 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:46.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:46.736 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:17.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:17.008 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:47.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:47.137 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:17.212 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:17.214 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:47.282 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:47.286 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:17.478 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:17.481 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:47.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:47.552 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:59:17.628 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:59:17.631 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:59:47.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:59:47.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:00:17.794 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:00:17.797 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:00:47.972 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:00:47.975 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:01:18.062 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:01:18.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:01:48.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:01:48.202 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:02:18.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:02:18.293 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:02:48.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:02:48.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:03:18.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:03:18.604 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:03:48.671 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:03:48.674 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:04:18.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:04:18.754 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:04:48.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:04:48.898 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:05:18.965 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:05:18.968 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:05:49.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:05:49.064 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:06:19.237 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:06:19.240 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:06:49.535 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:06:49.538 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:07:19.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:07:19.603 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:07:49.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:07:49.731 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:08:19.798 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:08:19.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:08:49.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:08:49.874 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:09:20.039 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:09:20.042 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:09:50.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:09:50.103 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:10:20.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:10:20.174 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:10:50.369 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:10:50.372 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:11:20.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:11:20.566 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:11:50.642 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:11:50.644 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:12:20.731 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:12:20.734 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:12:50.849 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:12:50.852 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:13:20.919 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:13:20.922 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:13:51.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:13:51.008 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:14:21.195 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:14:21.198 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:14:51.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:14:51.335 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:15:21.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:15:21.614 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:15:51.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:15:51.691 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:16:21.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:16:21.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:16:51.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:16:51.950 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:17:22.017 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:17:22.020 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:17:52.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:17:52.133 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:18:22.199 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:18:22.202 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:18:52.269 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:18:52.272 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:19:22.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:19:22.519 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:19:52.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:19:52.588 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:20:22.747 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:20:22.751 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:20:52.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:20:52.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:21:22.975 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:21:22.978 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:21:53.068 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:21:53.072 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:22:23.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:22:23.160 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:22:53.334 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:22:53.337 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:23:23.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:23:23.423 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:23:53.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:23:53.511 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:24:23.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:24:23.675 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:24:53.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:24:53.753 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:25:24.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:25:24.115 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:25:54.216 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:25:54.220 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:26:24.400 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:26:24.403 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:26:54.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:26:54.545 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:27:24.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:27:24.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:27:54.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:27:54.676 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:28:24.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:28:24.748 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:28:54.892 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:28:54.895 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:29:25.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:29:25.248 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:29:55.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:29:55.338 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:30:25.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:30:25.576 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:30:55.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:30:55.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:31:25.792 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:31:25.795 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:31:55.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:31:55.866 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:32:26.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:32:26.003 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:32:56.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:32:56.115 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:33:26.183 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:33:26.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:33:56.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:33:56.423 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:34:26.489 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:34:26.494 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:34:56.836 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:34:56.840 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:35:27.082 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:35:27.085 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:35:57.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:35:57.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:36:27.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:36:27.479 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:36:57.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:36:57.551 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:37:27.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:37:27.692 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:37:57.901 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:37:57.904 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:38:28.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:38:28.175 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:38:58.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:38:58.378 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:39:28.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:39:28.442 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:39:58.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:39:58.511 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:40:28.579 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:40:28.582 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:40:58.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:40:58.735 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:41:28.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:41:28.829 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:41:58.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:41:58.926 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:42:29.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:42:29.057 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:42:59.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:42:59.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:43:29.196 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:43:29.199 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:43:59.257 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:43:59.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:44:29.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:44:29.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:44:59.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:44:59.444 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:45:29.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:45:29.515 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:45:59.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:45:59.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:46:29.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:46:29.731 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:46:59.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:46:59.814 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:47:29.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:47:29.880 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:48:00.014 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:48:00.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:48:30.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:48:30.078 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:49:00.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:49:00.149 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:49:30.249 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:49:30.252 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:50:00.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:50:00.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:50:30.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:50:30.494 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:51:00.739 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:51:00.742 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:51:31.082 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:51:31.085 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:52:01.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:52:01.157 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:52:31.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:52:31.226 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:53:01.302 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:53:01.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:53:31.372 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:53:31.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:54:01.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:54:01.463 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:54:31.550 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:54:31.553 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:55:01.716 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:55:01.719 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:55:31.818 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:55:31.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:56:01.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:56:01.902 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:56:31.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:56:31.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:57:02.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:57:02.069 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:57:32.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:57:32.158 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:58:02.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:58:02.238 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:58:32.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:58:32.389 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:59:02.564 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:59:02.567 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:59:32.700 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T04:59:32.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:00:02.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:00:02.767 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:00:32.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:00:32.942 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:01:03.124 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:01:03.127 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:01:33.218 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:01:33.221 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:02:03.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:02:03.291 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:02:33.401 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:02:33.404 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:03:03.511 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:03:03.514 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:03:33.624 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:03:33.626 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:04:03.729 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:04:03.732 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:04:33.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:04:33.794 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:05:03.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:05:03.867 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:05:34.007 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:05:34.011 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:06:04.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:06:04.097 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:06:34.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:06:34.184 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:07:04.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:07:04.247 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:07:34.323 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:07:34.326 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:08:04.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:08:04.479 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:08:34.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:08:34.669 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:09:04.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:09:04.842 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:09:35.169 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:09:35.172 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:10:05.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:10:05.561 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:10:35.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:10:35.817 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:11:05.899 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:11:05.902 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:11:36.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:11:36.026 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:12:06.143 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:12:06.147 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:12:36.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:12:36.276 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:13:06.423 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:13:06.426 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:13:36.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:13:36.552 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:14:06.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:14:06.716 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:14:36.935 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:14:36.938 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:15:07.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:15:07.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:15:37.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:15:37.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:16:07.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:16:07.341 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:16:37.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:16:37.506 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:17:07.625 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:17:07.628 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:17:37.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:17:37.757 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:18:07.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:18:07.888 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:18:38.014 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:18:38.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:19:08.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:19:08.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:19:38.299 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:19:38.302 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:20:08.379 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:20:08.382 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:20:38.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:20:38.513 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:21:08.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:21:08.671 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:21:38.827 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:21:38.830 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:22:08.957 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:22:08.960 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:22:39.029 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:22:39.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:23:09.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:23:09.137 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:23:39.279 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:23:39.282 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:24:09.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:24:09.411 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:24:39.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:24:39.550 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:25:09.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:25:09.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:25:40.031 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:25:40.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:26:10.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:26:10.164 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:26:40.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:26:40.296 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:27:10.355 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:27:10.358 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:27:40.467 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:27:40.470 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:28:10.604 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:28:10.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:28:40.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:28:40.755 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:29:10.906 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:29:10.909 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:29:41.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:29:41.278 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:30:11.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:30:11.411 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:30:41.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:30:41.485 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:31:11.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:31:11.669 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:31:42.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:31:42.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:32:12.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:32:12.194 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:32:42.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:32:42.343 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:33:12.479 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:33:12.481 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:33:42.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:33:42.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:34:12.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:34:12.765 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:34:42.943 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:34:42.946 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:35:13.081 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:35:13.083 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:35:43.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:35:43.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:36:13.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:36:13.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:36:43.522 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:36:43.525 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:37:13.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:37:13.675 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:37:43.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:37:43.829 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:38:13.975 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:38:13.979 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:38:44.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:38:44.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:39:14.278 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:39:14.281 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:39:44.407 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:39:44.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:40:14.719 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:40:14.723 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:40:44.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:40:44.858 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:41:14.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:41:14.996 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:41:45.138 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:41:45.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:42:15.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:42:15.295 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:42:45.445 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:42:45.448 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:43:15.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:43:15.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:43:45.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:43:45.790 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:44:16.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:44:16.045 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:44:46.185 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:44:46.187 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:45:16.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:45:16.322 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:45:46.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:45:46.468 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:46:16.620 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:46:16.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:46:46.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:46:46.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:47:16.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:47:16.906 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:47:47.052 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:47:47.055 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:48:17.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:48:17.200 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:48:47.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:48:47.355 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:49:17.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:49:17.469 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:49:47.607 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:49:47.609 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:50:17.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:50:17.761 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:50:47.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:50:47.897 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:51:18.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:51:18.093 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:51:48.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:51:48.238 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:52:18.372 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:52:18.375 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:52:48.518 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:52:48.521 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:53:18.679 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:53:18.682 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:53:48.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:53:48.837 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:54:18.978 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:54:18.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:54:49.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:54:49.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:55:19.211 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:55:19.214 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:55:49.369 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:55:49.372 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:56:19.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:56:19.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:56:49.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:56:49.708 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:57:19.851 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:57:19.854 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:57:49.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:57:49.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:58:20.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:58:20.135 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:58:50.303 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:58:50.306 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:59:20.423 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:59:20.427 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:59:50.550 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T05:59:50.552 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:00:20.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:00:20.674 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:00:50.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:00:50.829 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:01:20.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:01:20.966 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:01:51.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:01:51.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:02:21.269 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:02:21.272 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:02:51.389 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:02:51.392 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:03:21.535 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:03:21.538 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:03:51.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:03:51.708 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:04:21.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:04:21.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:04:52.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:04:52.050 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:05:22.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:05:22.211 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:05:52.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:05:52.369 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:06:22.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:06:22.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:06:52.642 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:06:52.645 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:07:22.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:07:23.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:07:53.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:07:53.180 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:08:23.305 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:08:23.308 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:08:53.453 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:08:53.456 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:09:23.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:09:23.598 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:09:53.729 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:09:53.732 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:10:23.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:10:23.872 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:10:54.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:11:24.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:11:24.180 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:11:54.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:11:54.343 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:12:24.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:12:24.486 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:12:54.608 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:12:54.611 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:13:24.754 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:13:24.758 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:13:54.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:13:54.906 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:14:25.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:14:25.043 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:14:55.185 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:14:55.187 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:15:25.314 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:15:25.317 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:15:55.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:15:55.453 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:16:25.588 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:16:25.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:16:55.737 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:16:55.740 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:17:25.887 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:17:25.890 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:17:56.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:17:56.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:18:26.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:18:26.130 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:18:56.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:18:56.267 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:19:26.401 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:19:26.404 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:19:56.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:19:56.550 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:20:26.700 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:20:26.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:20:56.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:20:56.858 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:21:26.999 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:21:27.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:21:57.144 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:21:57.147 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:22:27.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:22:27.292 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:22:57.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:22:57.429 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:23:27.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:23:27.569 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:23:57.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:23:57.716 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:24:27.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:24:27.813 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:24:57.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:24:57.952 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:25:28.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:25:28.089 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:25:58.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:25:58.226 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:26:28.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:26:28.348 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:26:58.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:26:58.511 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:27:28.659 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:27:28.662 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:27:58.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:27:58.791 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:28:28.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:28:28.944 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:28:59.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:28:59.080 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:29:29.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:29:29.217 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:29:59.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:29:59.355 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:30:29.489 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:30:29.492 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:30:59.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:30:59.653 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:31:29.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:31:29.789 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:31:59.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:31:59.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:32:30.062 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:32:30.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:33:00.207 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:33:00.211 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:33:30.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:33:30.363 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:34:00.506 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:34:00.510 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:34:30.642 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:34:30.646 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:35:00.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:35:00.791 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:35:30.933 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:35:30.936 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:36:01.071 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:36:01.073 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:36:31.227 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:36:31.229 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:37:01.348 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:37:01.351 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:37:31.486 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:37:31.489 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:38:01.626 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:38:01.629 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:38:31.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:38:31.791 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:39:01.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:39:01.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:39:32.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:39:32.067 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:40:02.227 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:40:02.230 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:40:32.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:40:32.423 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:41:02.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:41:02.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:41:32.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:41:32.755 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:42:02.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:42:02.907 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:42:33.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:42:33.079 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:43:03.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:43:03.231 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:43:33.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:43:33.369 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:44:03.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:44:03.506 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:44:33.623 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:44:33.626 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:45:03.785 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:45:03.787 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:45:33.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:45:33.924 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:46:04.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:46:04.090 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:46:34.241 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:46:34.244 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:47:04.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:47:04.398 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:47:34.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:47:34.547 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:48:04.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:48:04.676 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:48:34.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:48:34.829 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:49:04.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:49:04.967 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:49:35.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:49:35.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:50:05.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:50:05.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:50:35.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:50:35.427 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:51:05.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:51:05.562 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:51:35.716 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:51:35.719 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:52:05.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:52:05.850 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:52:35.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:52:36.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:53:06.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:53:06.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:53:36.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:53:36.260 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:54:06.411 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:54:06.414 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:54:36.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:54:36.565 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:55:06.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:55:06.702 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:55:36.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:55:36.838 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:56:06.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:56:07.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:56:37.159 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:56:37.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:57:07.299 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:57:07.302 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:57:37.437 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:57:37.440 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:58:07.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:58:07.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:58:37.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:58:37.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:59:07.836 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:59:07.839 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:59:37.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T06:59:37.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:00:08.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:00:08.122 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:00:38.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:00:38.225 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:01:08.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:01:08.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:01:38.622 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:01:38.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:02:08.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:02:08.765 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:02:38.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:02:38.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:03:09.032 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:03:09.035 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:03:39.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:03:39.181 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:04:09.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:04:09.334 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:04:39.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:04:39.515 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:05:09.641 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:05:09.644 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:05:39.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:05:39.949 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:06:10.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:06:10.082 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:06:40.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:06:40.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:07:10.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:07:10.380 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:07:40.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:07:40.534 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:08:10.676 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:08:10.679 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:08:40.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:08:40.824 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:09:10.975 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:09:10.978 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:09:41.103 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:09:41.106 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:10:11.249 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:10:11.252 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:10:41.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:10:41.381 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:11:11.507 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:11:11.510 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:11:41.636 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:11:41.640 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:12:11.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:12:11.811 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:12:41.952 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:12:41.955 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:13:12.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:13:12.119 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:13:42.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:13:42.270 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:14:12.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:14:12.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:14:42.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:14:42.551 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:15:12.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:15:12.698 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:15:42.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:15:42.860 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:16:13.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:16:13.014 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:16:43.139 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:16:43.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:17:13.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:17:13.279 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:17:43.396 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:17:43.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:18:13.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:18:13.528 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:18:43.645 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:18:43.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:19:13.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:19:13.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:19:43.910 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:19:43.913 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:20:14.031 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:20:14.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:20:44.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:20:44.175 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:21:14.302 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:21:14.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:21:44.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:21:44.451 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:22:14.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:22:14.565 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:22:44.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:22:44.685 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:23:14.827 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:23:14.830 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:23:44.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:23:44.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:24:15.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:24:15.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:24:45.297 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:24:45.300 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:25:15.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:25:15.428 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:25:45.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:25:45.564 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:26:15.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:26:15.693 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:26:45.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:26:45.847 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:27:16.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:27:16.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:27:46.164 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:27:46.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:28:16.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:28:16.314 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:28:46.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:28:46.468 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:29:16.604 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:29:16.607 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:29:46.908 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:29:46.911 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:30:17.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:30:17.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:30:47.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:30:47.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:31:17.295 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:31:17.298 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:31:47.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:31:47.425 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:32:17.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:32:17.560 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:32:47.688 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:32:47.690 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:33:17.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:33:17.836 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:33:47.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:33:47.965 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:34:18.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:34:18.103 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:34:48.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:34:48.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:35:18.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:35:18.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:35:48.532 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:35:48.535 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:36:18.688 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:36:18.692 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:36:48.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:36:48.828 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:37:18.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:37:18.982 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:37:49.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:37:49.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:38:19.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:38:19.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:38:49.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:38:49.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:39:19.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:39:19.751 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:39:49.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:39:49.897 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:40:20.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:40:20.045 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:40:50.188 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:40:50.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:41:20.334 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:41:20.337 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:41:50.495 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:41:50.498 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:42:20.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:42:20.652 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:42:50.777 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:42:50.781 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:43:20.931 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:43:20.934 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:43:51.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:43:51.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:44:21.238 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:44:21.242 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:44:51.367 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:44:51.370 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:45:21.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:45:21.506 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:45:51.640 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:45:51.643 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:46:21.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:46:21.789 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:46:51.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:46:51.926 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:47:22.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:47:22.063 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:47:52.205 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:47:52.208 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:48:22.342 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:48:22.345 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:48:52.487 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:48:52.490 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:49:22.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:49:22.602 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:49:52.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:49:52.731 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:50:22.889 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:50:22.893 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:50:53.081 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:50:53.085 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:51:23.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:51:23.234 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:51:53.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:51:53.370 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:52:23.521 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:52:23.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:52:53.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:52:53.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:53:24.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:53:24.050 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:53:54.216 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:53:54.219 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:54:24.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:54:24.341 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:54:54.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:54:54.468 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:55:24.602 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:55:24.605 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:55:54.737 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:55:54.740 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:56:24.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:56:24.870 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:56:54.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:56:54.992 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:57:25.169 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:57:25.172 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:57:55.311 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:57:55.314 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:58:25.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:58:25.459 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:58:55.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:58:55.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:59:25.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:59:25.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:59:55.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T07:59:55.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:00:26.038 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:00:26.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:00:56.185 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:00:56.188 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:01:26.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:01:26.333 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:01:56.478 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:01:56.481 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:02:26.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:02:26.625 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:02:56.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:02:56.761 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:03:26.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:03:26.906 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:03:57.130 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:03:57.133 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:04:27.261 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:04:27.264 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:04:57.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:04:57.375 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:05:27.501 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:05:27.504 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:05:57.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:05:57.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:06:27.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:06:27.744 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:06:57.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:06:57.898 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:07:28.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:07:28.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:07:58.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:07:58.222 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:08:28.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:08:28.360 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:08:58.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:08:58.496 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:09:28.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:09:28.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:09:58.804 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:09:58.807 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:10:28.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:10:28.958 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:10:59.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:10:59.117 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:11:29.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:11:29.248 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:11:59.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:11:59.401 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:12:29.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:12:29.576 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:12:59.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:12:59.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:13:29.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:13:29.887 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:14:00.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:14:00.016 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:14:30.150 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:14:30.153 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:15:00.287 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:15:00.290 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:15:30.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:15:30.418 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:16:00.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:16:00.572 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:16:30.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:16:30.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:17:00.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:17:00.888 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:17:31.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:17:31.026 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:18:01.158 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:18:01.163 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:18:31.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:18:31.315 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:19:01.432 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:19:01.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:19:31.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:19:31.557 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:20:01.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:20:01.719 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:20:31.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:20:31.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:21:01.985 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:21:01.988 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:21:32.108 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:21:32.111 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:22:02.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:22:02.248 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:22:32.399 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:22:32.403 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:23:02.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:23:02.547 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:23:32.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:23:32.686 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:24:02.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:24:02.847 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:24:32.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:24:32.971 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:25:03.117 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:25:03.132 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:25:33.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:25:33.282 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:26:03.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:26:03.427 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:26:33.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:26:33.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:27:03.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:27:03.885 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:27:34.062 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:27:34.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:28:04.209 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:28:04.212 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:28:34.351 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:28:34.354 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:29:04.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:29:04.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:29:34.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:29:34.837 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:30:04.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:30:04.965 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:30:35.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:30:35.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:31:05.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:31:05.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:31:35.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:31:35.429 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:32:05.575 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:32:05.578 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:32:35.736 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:32:35.739 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:33:06.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:33:06.027 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:33:36.187 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:33:36.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:34:06.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:34:06.319 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:34:36.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:34:36.480 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:35:06.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:35:06.601 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:35:36.747 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:35:36.750 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:36:06.901 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:36:06.904 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:36:37.050 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:36:37.054 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:37:07.187 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:37:07.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:37:37.341 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:37:37.345 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:38:07.495 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:38:07.498 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:38:37.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:38:37.675 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:39:07.796 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:39:07.799 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:39:37.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:39:37.929 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:40:08.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:40:08.076 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:40:38.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:40:38.206 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:41:08.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:41:08.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:41:38.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:41:38.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:42:08.641 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:42:08.644 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:42:38.805 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:42:38.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:43:08.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:43:09.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:43:39.152 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:43:39.155 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:44:09.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:44:09.291 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:44:39.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:44:39.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:45:09.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:45:09.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:45:39.724 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:45:39.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:46:09.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:46:09.881 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:46:40.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:46:40.042 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:47:10.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:47:10.196 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:47:40.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:47:40.342 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:48:10.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:48:10.453 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:48:40.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:48:40.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:49:10.674 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:49:10.677 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:49:40.829 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:49:40.833 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:50:10.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:50:10.953 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:50:41.085 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:50:41.088 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:51:11.198 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:51:11.201 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:51:41.368 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:51:41.371 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:52:11.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:52:11.516 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:52:41.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:52:41.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:53:11.812 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:53:11.815 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:53:41.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:53:41.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:54:12.044 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:54:12.047 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:54:42.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:54:42.169 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:55:12.302 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:55:12.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:55:42.448 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:55:42.451 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:56:12.576 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:56:12.580 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:56:42.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:56:42.716 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:57:12.866 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:57:12.869 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:57:42.986 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:57:42.989 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:58:13.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:58:13.159 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:58:43.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:58:43.313 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:59:13.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:59:13.459 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:59:43.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T08:59:43.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:00:13.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:00:13.716 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:00:43.859 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:00:43.862 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:01:14.012 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:01:14.015 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:01:44.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:01:44.169 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:02:14.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:02:14.309 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:02:44.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:02:44.427 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:03:14.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:03:14.572 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:03:44.706 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:03:44.710 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:04:14.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:04:14.846 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:04:44.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:04:44.966 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:05:15.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:05:15.103 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:05:45.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:05:45.256 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:06:15.399 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:06:15.402 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:06:45.528 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:06:45.531 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:07:15.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:07:15.692 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:07:45.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:07:45.829 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:08:15.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:08:15.949 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:08:46.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:08:46.011 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:09:16.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:09:16.158 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:09:46.283 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:09:46.286 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:10:16.411 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:10:16.414 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:10:46.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:10:46.576 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:11:16.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:11:16.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:11:46.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:11:46.871 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:12:16.992 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:12:16.995 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:12:47.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:12:47.156 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:13:17.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:13:17.295 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:13:47.446 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:13:47.449 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:14:17.801 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:14:17.804 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:14:47.927 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:14:47.930 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:15:18.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:15:18.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:15:48.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:15:48.195 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:16:18.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:16:18.266 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:16:48.399 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:16:48.402 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:17:18.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:17:18.532 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:17:48.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:17:48.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:18:18.830 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:18:18.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:18:48.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:18:48.971 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:19:19.102 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:19:19.105 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:19:49.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:19:49.267 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:20:19.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:20:19.418 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:20:49.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:20:49.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:21:19.674 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:21:19.677 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:21:49.828 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:21:49.831 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:22:20.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:22:20.004 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:22:50.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:22:50.139 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:23:20.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:23:20.431 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:23:50.534 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:23:50.537 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:24:20.663 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:24:20.666 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:24:50.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:24:50.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:25:20.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:25:20.931 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:25:51.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:25:51.093 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:26:21.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:26:21.231 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:26:51.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:26:51.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:27:21.523 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:27:21.526 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:27:51.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:27:51.675 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:28:21.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:28:21.828 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:28:51.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:28:51.982 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:29:22.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:29:22.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:29:52.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:29:52.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:30:22.416 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:30:22.419 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:30:52.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:30:52.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:31:22.714 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:31:22.717 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:31:52.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:31:52.870 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:32:23.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:32:23.024 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:32:53.159 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:32:53.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:33:23.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:33:23.341 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:33:53.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:33:53.494 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:34:23.644 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:34:23.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:34:53.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:34:53.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:35:23.943 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:35:23.947 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:35:54.098 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:35:54.101 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:36:24.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:36:24.237 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:36:54.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:36:54.365 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:37:24.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:37:24.519 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:37:54.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:37:54.664 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:38:24.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:38:24.818 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:38:54.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:38:54.947 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:39:25.092 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:39:25.095 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:39:55.227 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:39:55.230 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:40:25.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:40:25.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:40:55.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:40:55.528 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:41:25.670 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:41:25.673 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:41:55.796 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:41:55.799 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:42:25.936 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:42:25.939 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:42:56.097 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:42:56.100 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:43:26.251 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:43:26.254 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:43:56.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:43:56.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:44:26.550 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:44:26.553 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:44:56.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:44:56.694 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:45:26.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:45:26.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:45:56.909 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:45:56.912 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:46:27.049 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:46:27.052 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:46:57.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:46:57.205 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:47:27.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:47:27.359 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:47:57.495 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:47:57.498 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:48:27.638 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:48:27.641 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:48:57.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:48:57.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:49:27.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:49:27.881 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:49:58.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:49:58.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:50:28.136 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:50:28.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:50:58.257 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:50:58.260 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:51:28.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:51:28.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:51:58.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:51:58.557 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:52:28.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:52:28.687 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:52:58.829 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:52:58.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:53:28.974 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:53:28.977 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:53:59.102 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:53:59.105 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:54:29.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:54:29.251 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:54:59.379 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:54:59.382 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:55:29.506 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:55:29.509 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:55:59.646 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:55:59.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:56:29.827 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:56:29.829 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:56:59.973 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:56:59.976 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:57:30.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:57:30.140 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:58:00.282 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:58:00.284 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:58:30.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:58:30.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:59:00.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:59:00.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:59:30.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T09:59:30.737 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:00:00.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:00:00.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:00:31.016 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:00:31.019 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:01:01.162 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:01:01.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:01:31.249 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:01:31.252 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:02:01.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:02:01.389 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:02:31.548 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:02:31.551 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:03:01.721 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:03:01.724 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:03:31.875 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:03:31.878 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:04:02.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:04:02.011 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:04:32.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:04:32.194 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:05:02.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:05:02.338 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:05:32.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:05:32.481 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:06:02.572 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:06:02.575 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:06:32.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:06:32.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:07:02.872 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:07:02.875 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:07:33.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:07:33.037 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:08:03.302 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:08:03.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:08:33.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:08:33.427 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:09:03.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:09:03.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:09:33.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:09:33.720 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:10:03.854 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:10:03.857 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:10:33.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:10:33.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:11:04.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:11:04.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:11:34.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:11:34.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:12:04.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:12:04.417 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:12:34.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:12:34.574 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:13:04.707 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:13:04.710 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:13:34.865 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:13:34.868 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:14:04.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:14:05.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:14:35.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:14:35.125 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:15:05.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:15:05.283 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:15:35.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:15:35.428 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:16:05.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:16:05.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:16:35.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:16:35.702 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:17:05.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:17:05.847 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:17:35.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:17:35.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:18:06.138 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:18:06.141 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:18:36.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:18:36.299 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:19:06.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:19:06.417 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:19:36.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:19:36.555 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:20:06.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:20:06.686 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:20:36.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:20:36.822 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:21:06.992 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:21:06.995 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:21:37.255 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:21:37.259 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:22:07.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:22:07.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:22:37.550 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:22:37.553 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:23:07.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:23:07.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:23:37.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:23:37.827 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:24:07.972 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:24:07.975 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:24:38.115 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:24:38.118 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:25:08.256 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:25:08.258 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:25:38.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:25:38.397 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:26:08.757 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:26:08.760 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:26:38.903 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:26:38.906 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:27:09.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:27:09.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:27:39.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:27:39.206 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:28:09.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:28:09.386 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:28:39.514 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:28:39.517 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:29:09.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:29:09.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:29:39.818 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:29:39.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:30:09.971 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:30:09.974 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:30:40.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:30:40.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:31:10.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:31:10.283 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:31:40.432 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:31:40.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:32:10.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:32:10.572 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:32:40.715 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:32:40.718 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:33:10.865 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:33:10.868 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:33:40.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:33:41.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:34:11.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:34:11.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:34:41.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:34:41.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:35:11.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:35:11.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:35:41.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:35:41.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:36:11.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:36:11.651 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:36:41.794 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:36:41.796 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:37:11.931 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:37:11.934 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:37:42.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:37:42.087 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:38:12.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:38:12.224 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:38:42.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:38:42.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:39:12.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:39:12.508 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:39:42.633 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:39:42.636 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:40:12.753 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:40:12.757 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:40:42.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:40:42.894 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:41:13.026 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:41:13.029 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:41:43.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:41:43.184 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:42:13.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:42:13.339 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:42:43.489 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:42:43.492 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:43:13.620 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:43:13.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:43:43.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:43:43.765 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:44:13.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:44:13.903 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:44:44.044 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:44:44.047 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:45:14.115 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:45:14.118 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:45:44.260 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:45:44.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:46:14.399 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:46:14.402 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:46:44.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:46:44.562 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:47:14.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:47:14.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:47:44.841 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:47:44.844 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:48:14.978 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:48:14.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:48:45.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:48:45.117 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:49:15.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:49:15.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:49:45.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:49:45.384 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:50:15.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:50:15.512 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:50:45.638 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:50:45.641 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:51:15.779 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:51:15.782 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:51:45.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:51:45.903 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:52:16.072 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:52:16.075 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:52:46.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:52:46.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:53:16.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:53:16.384 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:53:46.535 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:53:46.538 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:54:16.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:54:16.694 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:54:46.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:54:46.837 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:55:17.017 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:55:17.020 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:55:47.314 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:55:47.316 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:56:17.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:56:17.446 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:56:47.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:56:47.576 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:57:17.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:57:17.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:57:47.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:57:47.825 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:58:17.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:58:17.961 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:58:48.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:58:48.099 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:59:18.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:59:18.236 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:59:48.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T10:59:48.389 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:00:18.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:00:18.544 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:00:48.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:00:48.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:01:18.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:01:18.842 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:01:48.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:01:48.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:02:19.138 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:02:19.141 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:02:49.309 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:02:49.312 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:03:19.437 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:03:19.443 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:03:49.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:03:49.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:04:19.737 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:04:19.739 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:04:49.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:04:49.886 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:05:20.038 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:05:20.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:05:50.159 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:05:50.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:06:20.309 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:06:20.313 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:06:50.453 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:06:50.456 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:07:20.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:07:20.603 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:07:50.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:07:50.774 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:08:20.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:08:20.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:08:51.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:08:51.080 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:09:21.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:09:21.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:09:51.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:09:51.385 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:10:21.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:10:21.548 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:10:51.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:10:51.694 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:11:21.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:11:21.823 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:11:51.950 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:11:51.953 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:12:22.103 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:12:22.106 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:12:52.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:12:52.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:13:22.393 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:13:22.396 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:13:52.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:13:52.547 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:14:22.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:14:22.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:14:52.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:14:52.841 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:15:22.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:15:22.962 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:15:53.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:15:53.110 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:16:23.256 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:16:23.260 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:16:53.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:16:53.393 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:17:23.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:17:23.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:17:53.693 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:17:53.695 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:18:23.854 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:18:23.857 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:18:54.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:18:54.010 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:19:24.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:19:24.157 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:19:54.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:19:54.318 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:20:24.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:20:24.468 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:20:54.622 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:20:54.626 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:21:24.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:21:24.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:21:55.046 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:21:55.049 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:22:25.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:22:25.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:22:55.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:22:55.335 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:23:25.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:23:25.459 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:23:55.602 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:23:55.605 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:24:25.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:24:25.751 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:24:55.893 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:24:55.896 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:25:25.972 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:25:25.975 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:25:56.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:25:56.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:26:26.278 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:26:26.281 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:26:56.423 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:26:56.426 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:27:26.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:27:26.547 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:27:56.715 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:27:56.719 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:28:26.851 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:28:26.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:28:56.994 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:28:56.998 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:29:27.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:29:27.137 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:29:57.287 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:29:57.290 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:30:27.423 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:30:27.426 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:30:57.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:30:57.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:31:27.706 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:31:27.709 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:31:57.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:31:57.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:32:27.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:32:27.993 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:32:58.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:32:58.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:33:28.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:33:28.251 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:33:58.434 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:33:58.437 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:34:28.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:34:28.592 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:34:58.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:34:58.728 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:35:28.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:35:28.874 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:35:59.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:35:59.004 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:36:29.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:36:29.164 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:36:59.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:36:59.316 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:37:29.451 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:37:29.454 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:37:59.596 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:37:59.599 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:38:29.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:38:29.736 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:38:59.888 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:38:59.890 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:39:30.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:39:30.027 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:40:00.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:40:00.168 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:40:30.299 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:40:30.302 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:41:00.443 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:41:00.446 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:41:30.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:41:30.617 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:42:00.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:42:00.754 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:42:30.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:42:30.907 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:43:01.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:43:01.061 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:43:31.187 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:43:31.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:44:01.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:44:01.269 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:44:31.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:44:31.397 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:45:01.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:45:01.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:45:31.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:45:31.687 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:46:01.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:46:01.841 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:46:31.992 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:46:31.995 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:47:02.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:47:02.140 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:47:32.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:47:32.295 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:48:02.453 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:48:02.457 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:48:32.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:48:32.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:49:02.736 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:49:02.740 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:49:32.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:49:32.817 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:50:02.942 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:50:02.946 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:50:33.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:50:33.067 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:51:03.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:51:03.194 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:51:33.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:51:33.315 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:52:03.432 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:52:03.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:52:33.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:52:33.564 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:53:03.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:53:03.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:53:33.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:53:33.854 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:54:03.972 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:54:03.975 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:54:34.101 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:54:34.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:55:04.243 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:55:04.246 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:55:34.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:55:34.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:56:04.537 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:56:04.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:56:34.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:56:34.652 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:57:04.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:57:04.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:57:34.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:57:34.925 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:58:05.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:58:05.057 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:58:35.184 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:58:35.187 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:59:05.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:59:05.328 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:59:35.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T11:59:35.457 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:00:05.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:00:05.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:00:35.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:00:35.774 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:01:05.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:01:05.918 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:01:36.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:01:36.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:02:06.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:02:06.225 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:02:36.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:02:36.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:03:06.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:03:06.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:03:36.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:03:36.693 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:04:06.804 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:04:06.807 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:04:36.934 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:04:36.937 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:05:07.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:05:07.098 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:05:37.234 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:05:37.238 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:06:07.375 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:06:07.378 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:06:37.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:06:37.515 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:07:07.626 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:07:07.628 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:07:37.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:07:37.765 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:08:07.907 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:08:07.911 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:08:38.044 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:08:38.047 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:09:08.140 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:09:08.143 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:09:38.260 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:09:38.263 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:10:08.389 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:10:08.392 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:10:38.517 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:10:38.521 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:11:08.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:11:08.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:11:38.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:11:38.778 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:12:08.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:12:08.925 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:12:39.049 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:12:39.052 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:13:09.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:13:09.213 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:13:39.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:13:39.364 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:14:09.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:14:09.505 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:14:39.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:14:39.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:15:09.792 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:15:09.795 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:15:39.930 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:15:39.933 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:16:10.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:16:10.089 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:16:40.238 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:16:40.241 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:17:10.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:17:10.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:17:40.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:17:40.654 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:18:10.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:18:10.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:18:40.942 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:18:40.946 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:19:11.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:19:11.076 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:19:41.200 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:19:41.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:20:11.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:20:11.348 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:20:41.496 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:20:41.499 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:21:11.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:21:11.646 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:21:41.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:21:41.776 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:22:11.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:22:11.957 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:22:42.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:22:42.117 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:23:12.251 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:23:12.254 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:23:42.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:23:42.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:24:12.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:24:12.550 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:24:42.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:24:42.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:25:12.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:25:12.827 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:25:42.980 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:25:42.983 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:26:13.131 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:26:13.134 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:26:43.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:26:43.307 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:27:13.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:27:13.442 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:27:43.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:27:43.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:28:13.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:28:13.728 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:28:43.892 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:28:43.895 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:29:14.038 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:29:14.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:29:44.175 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:29:44.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:30:14.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:30:14.319 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:30:44.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:30:44.460 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:31:14.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:31:14.614 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:31:44.743 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:31:44.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:32:14.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:32:14.885 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:32:45.038 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:32:45.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:33:15.175 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:33:15.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:33:45.321 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:33:45.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:34:15.467 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:34:15.469 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:34:45.620 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:34:45.622 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:35:15.757 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:35:15.759 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:35:45.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:35:45.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:36:16.056 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:36:16.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:36:46.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:36:46.231 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:37:16.354 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:37:16.358 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:37:46.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:37:46.503 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:38:16.654 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:38:16.660 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:38:46.816 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:38:46.819 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:39:16.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:39:16.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:39:47.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:39:47.092 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:40:17.399 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:40:17.402 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:40:47.698 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:40:47.700 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:41:17.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:41:17.828 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:41:47.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:41:47.958 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:42:18.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:42:18.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:42:48.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:42:48.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:43:18.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:43:18.394 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:43:48.499 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:43:48.502 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:44:18.640 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:44:18.643 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:44:48.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:44:48.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:45:18.931 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:45:18.934 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:45:49.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:45:49.087 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:46:19.205 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:46:19.208 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:46:49.333 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:46:49.335 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:47:19.478 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:47:19.481 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:47:49.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:47:49.619 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:48:19.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:48:19.776 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:48:49.898 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:48:49.901 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:49:20.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:49:20.030 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:49:50.173 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:49:50.176 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:50:20.348 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:50:20.351 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:50:50.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:50:50.487 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:51:20.633 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:51:20.636 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:51:50.760 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:51:50.763 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:52:20.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:52:20.908 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:52:51.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:52:51.030 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:53:21.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:53:21.175 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:53:51.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:53:51.472 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:54:21.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:54:21.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:54:51.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:54:51.903 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:55:22.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:55:22.047 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:55:52.174 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:55:52.177 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:56:22.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:56:22.322 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:56:52.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:56:52.466 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:57:22.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:57:22.604 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:57:52.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:57:52.751 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:58:22.899 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:58:22.902 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:58:53.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:58:53.069 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:59:23.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:59:23.218 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:59:53.367 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T12:59:53.370 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:00:23.497 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:00:23.500 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:00:53.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:00:53.654 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:01:23.804 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:01:23.807 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:01:53.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:01:53.947 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:02:24.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:02:24.093 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:02:54.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:02:54.248 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:03:24.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:03:24.389 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:03:54.514 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:03:54.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:04:24.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:04:24.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:04:54.805 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:04:54.808 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:05:24.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:05:24.945 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:05:55.115 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:05:55.118 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:06:25.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:06:25.269 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:06:55.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:06:55.338 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:07:25.472 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:07:25.475 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:07:55.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:07:55.620 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:08:25.756 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:08:25.759 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:08:55.919 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:08:55.921 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:09:26.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:09:26.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:09:56.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:09:56.181 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:10:26.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:10:26.340 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:10:56.473 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:10:56.476 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:11:26.627 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:11:26.630 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:11:56.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:11:56.792 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:12:26.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:12:26.949 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:12:57.088 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:12:57.091 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:13:27.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:13:27.269 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:13:57.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:13:57.412 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:14:27.540 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:14:27.543 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:14:57.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:14:57.697 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:15:27.840 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:15:27.842 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:15:57.977 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:15:57.993 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:16:28.147 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:16:28.150 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:16:58.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:16:58.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:17:28.393 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:17:28.396 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:17:58.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:17:58.526 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:18:28.703 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:18:28.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:18:58.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:18:58.851 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:19:29.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:19:29.122 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:19:59.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:19:59.260 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:20:29.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:20:29.412 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:20:59.563 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:20:59.565 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:21:29.700 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:21:29.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:21:59.853 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:21:59.856 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:22:29.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:22:29.994 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:23:00.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:23:00.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:23:30.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:23:30.266 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:24:00.403 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:24:00.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:24:30.533 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:24:30.536 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:25:00.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:25:00.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:25:30.795 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:25:30.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:26:00.956 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:26:00.960 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:26:31.102 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:26:31.105 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:27:01.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:27:01.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:27:31.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:27:31.388 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:28:01.505 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:28:01.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:28:31.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:28:31.825 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:29:02.003 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:29:02.006 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:29:32.149 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:29:32.151 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:30:02.295 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:30:02.298 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:30:32.432 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:30:32.436 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:31:02.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:31:02.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:31:32.772 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:31:32.775 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:32:02.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:32:02.908 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:32:33.043 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:32:33.047 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:33:03.183 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:33:03.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:33:33.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:33:33.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:34:03.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:34:03.460 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:34:33.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:34:33.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:35:03.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:35:03.867 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:35:33.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:35:33.996 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:36:04.147 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:36:04.150 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:36:34.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:36:34.279 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:37:04.403 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:37:04.407 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:37:34.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:37:34.544 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:38:04.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:38:04.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:38:34.859 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:38:34.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:39:05.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:39:05.022 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:39:35.147 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:39:35.151 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:40:05.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:40:05.279 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:40:35.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:40:35.423 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:41:05.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:41:05.560 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:41:35.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:41:35.708 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:42:05.892 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:42:05.895 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:42:36.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:42:36.030 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:43:06.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:43:06.158 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:43:36.290 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:43:36.293 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:44:06.395 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:44:06.397 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:44:36.534 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:44:36.537 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:45:06.670 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:45:06.673 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:45:36.817 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:45:36.820 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:46:07.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:46:07.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:46:37.198 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:46:37.201 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:47:07.326 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:47:07.329 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:47:37.455 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:47:37.457 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:48:07.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:48:07.594 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:48:37.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:48:37.732 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:49:07.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:49:07.861 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:49:38.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:49:38.013 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:50:08.148 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:50:08.151 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:50:38.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:50:38.226 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:51:08.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:51:08.359 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:51:38.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:51:38.488 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:52:08.613 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:52:08.616 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:52:38.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:52:38.745 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:53:08.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:53:08.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:53:39.032 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:53:39.035 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:54:09.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:54:09.180 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:54:39.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:54:39.317 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:55:09.468 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:55:09.471 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:55:39.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:55:39.532 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:56:09.659 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:56:09.661 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:56:39.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:56:39.774 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:57:09.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:57:09.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:57:39.986 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:57:39.989 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:58:10.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:58:10.157 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:58:40.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:58:40.322 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:59:10.470 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:59:10.473 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:59:40.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T13:59:40.604 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:00:10.739 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:00:10.742 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:00:40.908 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:00:40.911 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:01:11.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:01:11.048 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:01:41.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:01:41.194 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:02:11.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:02:11.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:02:41.498 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:02:41.500 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:03:11.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:03:11.659 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:03:41.813 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:03:41.816 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:04:11.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:04:11.963 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:04:42.104 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:04:42.107 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:05:12.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:05:12.253 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:05:42.403 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:05:42.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T14:05:52.510 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-07-06T14:05:52.512 DEBUG:teuthology.task.console_log:Killing console logger for smithi089 2023-07-06T14:05:52.512 DEBUG:teuthology.task.console_log:Killing console logger for smithi172 2023-07-06T14:05:52.513 DEBUG:teuthology.exit:Finished running handlers